Meeting with DjonDB
Hello, I work on relant, I would like to tell you about my experience with DjonDB, and what was the task I faced the task and why we decided to use a NoSQL solution, especially DjonDB.
We are engaged in the implementation of IPTV services, and It was necessary to organize the server monitoring signal quality or the IPTV servers. On the servers was a program that could send the statistics of the signal and each channel on the monitor server in JSON format. Of course, we also have syslog messages. But the most informative messages are received in JSON.For example:
and others like its.
So, We needed to store and parse JSON, determine relationships of tables to prevent problems with the constant changing of the message format, as the software is constantly changing and the message format may change as well, these changes may require additional fields and messages. Also, We can add new dvb adapters or the number of channels, and it is not associated with the update software.
Do I have to parse JSON? Is it comfortable to work with it on the client side? For us It is better to use a database with native support for JSON :) So, using a database that supports JSON natively will solve the problems of the constantly changing format, because the messages will fall directly to the database without any modification. The other problem was with the queries, which will need to change everytime the message format changed, using a NoSQL solution allow us to work faster because we don't need to change our code everytime the message changed. Less work to do.
There are many NoSQL solutions out there, but as it turned out, not all are welll suited for us. Limited to the MongoDB database of 4GB for 32-bit systems, and it is not resistant to falls, CouchDB requires knowledge about the content of messages. We could use Redis, but we found it much easier to work with DjonDB :) it's resistant to crashes, runs on 32-bit systems, fast, finally it allows you to retrieve only the fields you require in a very easy way (that's all about djondb, easier to use). And All in one.
In fact, We won! Intuitive query language and native support to JSON documents on Djondb makes working with the monitoring system transparent and straighforward. I made interactive graphics display of signal quality using morris.js. Work on them was very clear. I just asked for the documents to DjonDB and send it to morris in the browser:
Of course, there could be an «injection» code attack, but in this case, the server of monitoring knows where It can receive messages and trusts them, so it's not a big deal. In the ideal case, you must, of course, filter the data for the expected data type.
One problem that started to emerge with our first design was that the logging started to grow up very fast and we actually didn't required logs too old to the graphs, then Juan Pablo Crossley suggested solution to the problem. The solution was simple like all genius :). We just changed to namespaces based on the date and that make very easy to remove old records and focus on the real data we are requiring on our graphics report.
While I led the development I have questions and difficult situations, but community help me a lot. DjonDB project is progressing very fast, and it has a stupendous support!
Next in line next project and I'm getting ready to re-use DjonDB. Join now!
Hello, I work on relant, I would like to tell you about my experience with DjonDB, and what was the task I faced the task and why we decided to use a NoSQL solution, especially DjonDB.
We are engaged in the implementation of IPTV services, and It was necessary to organize the server monitoring signal quality or the IPTV servers. On the servers was a program that could send the statistics of the signal and each channel on the monitor server in JSON format. Of course, we also have syslog messages. But the most informative messages are received in JSON.For example:
{"stream":"Stream#1",
"type":"channel",
"bitrate":935936,
"output":"udp:\/\/@239.255.1.2:1014",
"server":"iptv",
"ready":true,
"channel":"Channel2",
"time":1361696024}
{"stream":"Stream#1",
"adapter":1,
"type":"dvb",
"bitrate":14710784,
"lock":true,
"server":"iptv",
"unc":0,
"snr":59,
"ber":0,
"time":1361695904}
"type":"channel",
"bitrate":935936,
"output":"udp:\/\/@239.255.1.2:1014",
"server":"iptv",
"ready":true,
"channel":"Channel2",
"time":1361696024}
{"stream":"Stream#1",
"adapter":1,
"type":"dvb",
"bitrate":14710784,
"lock":true,
"server":"iptv",
"unc":0,
"snr":59,
"ber":0,
"time":1361695904}
and others like its.
So, We needed to store and parse JSON, determine relationships of tables to prevent problems with the constant changing of the message format, as the software is constantly changing and the message format may change as well, these changes may require additional fields and messages. Also, We can add new dvb adapters or the number of channels, and it is not associated with the update software.
Do I have to parse JSON? Is it comfortable to work with it on the client side? For us It is better to use a database with native support for JSON :) So, using a database that supports JSON natively will solve the problems of the constantly changing format, because the messages will fall directly to the database without any modification. The other problem was with the queries, which will need to change everytime the message format changed, using a NoSQL solution allow us to work faster because we don't need to change our code everytime the message changed. Less work to do.
There are many NoSQL solutions out there, but as it turned out, not all are welll suited for us. Limited to the MongoDB database of 4GB for 32-bit systems, and it is not resistant to falls, CouchDB requires knowledge about the content of messages. We could use Redis, but we found it much easier to work with DjonDB :) it's resistant to crashes, runs on 32-bit systems, fast, finally it allows you to retrieve only the fields you require in a very easy way (that's all about djondb, easier to use). And All in one.
In fact, We won! Intuitive query language and native support to JSON documents on Djondb makes working with the monitoring system transparent and straighforward. I made interactive graphics display of signal quality using morris.js. Work on them was very clear. I just asked for the documents to DjonDB and send it to morris in the browser:
$hret=$this->djc->find('astra', 'astralog'.$lastday, '$"stream","$"unc",$"ber",$"snr",$"time"', '$"stream"=="Stream#1" and $"type"=="dvb" and $"time">'.$lasthour);
for ($i = 0; $i < $hret->size(); $i++) {
if ($i!=0){
$result.=',';
}
$result.=$hret->get($i)->toChar();
}
$result.=']';
for ($i = 0; $i < $hret->size(); $i++) {
if ($i!=0){
$result.=',';
}
$result.=$hret->get($i)->toChar();
}
$result.=']';
Of course, there could be an «injection» code attack, but in this case, the server of monitoring knows where It can receive messages and trusts them, so it's not a big deal. In the ideal case, you must, of course, filter the data for the expected data type.
One problem that started to emerge with our first design was that the logging started to grow up very fast and we actually didn't required logs too old to the graphs, then Juan Pablo Crossley suggested solution to the problem. The solution was simple like all genius :). We just changed to namespaces based on the date and that make very easy to remove old records and focus on the real data we are requiring on our graphics report.
While I led the development I have questions and difficult situations, but community help me a lot. DjonDB project is progressing very fast, and it has a stupendous support!
Next in line next project and I'm getting ready to re-use DjonDB. Join now!