Feature #311

Log serialization format enhancement for drawing graphs

Added by Baud Rémy over 7 years ago. Updated about 4 years ago.

Status:NewStart date:02/21/2011
Priority:NormalDue date:
Assignee:-% Done:

0%

Category:MONITOR
Target version:-
Sponsor: Ergonomic impact:
Functional impact:

Description

This is a performance issue about how the logs and aggLogs are serialized from the Monitor's servlet to Highcharts.

For info: we use in ExtJs a specific class called "Ext.data.JsonStore" in a restfull manner that permits an efficient high-level retrivial and edition of data. To use this nice feature, JS objects have to be serialized in the way that this class can read id. Here is how its done:

Example for job's daily log:


{
   "success":true,
   "message":"Les logs ont bien été récupérés",
   "noPagingCount":132,
   "data":[
            {
             "time":"2011-02-21 10:58:32",
             "delay":1.1390000581741333,
             etc...
            },{
             "time":"2011-02-21 10:53:32",
             "delay":1.6050000190734863,
             etc...
            }
           ]
}

The property success tells if the action has been performed successfully on the server side, and is used to display an error message if there was a problem. The column "noPagingCount" contains info about the pagination. The section data contains the logs values and we see here that each log object is serialized one after one.

Now lets see how we feed these data into Highcharts to create graphs.:

Here is how we can declare an Highcharts graph:


series: [{
          serieName: 'time',
          value: [2011-02-21 10:58:32, 2011-02-21 10:58:35, ...]
    },{
              serieName: 'delay',
              value: [1.605, 1.805, ...]
        }]

As we can see here, there are no more log objects, but series of specific fields of the log collection. What we do to create the graph is polling over each log in the json store and add to the graph serie the "time" or "delay" value.

It would be far more efficient to retrieve directly the log series from the server.

something like:


{
   "success":true,
   "message":"Les logs ont bien été récupérés",
   "noPagingCount":132,
   "data":[
            {
             "serieName": "time",
             "value":[2011-02-21 10:58:32, 2011-02-21 10:58:35, ...],
            },{
             "serieName":"delay",
             "value":[1.605, 1.805, ...],
             etc...
            }
           ]
}

What to expect:
+ less data overhead because column title are not repeated
+ less computing time because we dont convert object's to series in js afterward

Implementation:
We need the log collection like it is now because its used in maintainance. (delete raw logs..)
This feature could be achieved by making a new endpoint to retrieve log series or by making a different view of the log collection.

History

#1 Updated by Baud Rémy over 7 years ago

  • Assignee set to Baud Rémy

#2 Updated by Mérour Xavier about 7 years ago

  • Assignee deleted (Baud Rémy)

#3 Updated by Magoni Bruno over 4 years ago

  • Project changed from MONITOR to easySDI
  • Category deleted (Java servlet)
  • Target version deleted (Future)

#4 Updated by Magoni Bruno about 4 years ago

  • Category set to MONITOR

Also available in: Atom PDF