Logship Logo

Documentation

The how to guide on logship.

  • kusto count

    The count expresion will perform a count of all incoming rows. Returning a single column ‘count’.

  • kusto distinct

    The distinct operator will aggregate all of the distinct values of the given columns.

  • kusto limit / take

    The take expresion limits the number of results returned in a query. It can be used anywhere within the query pipeline.

  • kusto ago operator

    The ago operator returns a datetime of ‘time range’ in the past.

  • kusto bin operator

    The round operator floors values by a given interval.

  • kusto orderby

    ```kusto table | order by column1 ASC, column2 DESC, column3

  • kusto project

    ```kusto table | project column1, column2, column3 | project column1

  • kusto where

    table
    | where intColumn > 1500
    | where intColumn == 8
    | where stringColumn contains "abc"
    
  • Logship Charts Basics

    The basics of creating a chart using logship charts!

    Overview

    Logship charting uses Kusto queries over the backend metris or logs data to produce charts.

    Logship Charts screenshot

    Charts are built using queries like this one:

    [proc.stat.cpu] 
    | where PreciseTimestamp > {startTime}
    | where PreciseTimestamp < {endTime}
    | where processname contains 'logship'
    | project PreciseTimestamp, machinename, processname, average
    

    The charts will graph the first available “number” value in your query result (‘average’ in the example above). And MUST contain the “PreciseTimestamp” field. Each unique set of tags will graph an individual series, or line on your chart.

    Paramaterization

    Parameters allow you to link queries to controllable fields on the UI. The {startTime} and {endTime} values will always be available, and are tied to the time controls on the metrics page.

  • compacted file size logger

    The compacted file size logger service emits metrics on the total size, count, and number of rows of files within the entire system.

    Details

    The Service is responsible for emitting 3 metrics:

    Metric Value
    streaming.compactedfiles.totalSize The total size of all compacted files by account / schema.
    streaming.compactedfiles.totalRecords The total records count of all compacted files by account / schema.
    streaming.compactedfiles.count The number of all compacted files by account / schema.

    The configuration for this service:

    {
      "logs": {
        "compacted-file-stats": {
          "Enabled": true,
          "Interval": "00:01:00",
        }
      }
    }
    
  • metric time range compactor

    The time range compactor is in charge of aggregating metrics over shorter time range intervals.

    Details

    As raw metrics come in, they are either 1 value per metric, or pre-aggregated by the agents. The time range compactor would take all metrics over a particular time range, and bucket them over 5 minute intervals. Saving space, and improving query times.

    The Service is responsible for emitting 3 metrics:

    Metric Value
    metrics.compaction.exectiontime The total execution time of the compaction task.

    The configuration for this service:

    {
      "metrics": {
        "time-range-compaction": {
          "Enabled": true, // Enabled by default.
          "Interval": "00:05:00", // How often the compaction task runs. (5min default)
        }
      }
    }
    
  • Agent Development

    Install Rust

  • Agent Configuration

    The logship agent is configured via json document, it won’t start without a config. Provide a filepath as the agent’s first argument to specify a configuration:

    agent-rust.exe ./path/to/config.json
    
  • kusto summarize

    ```kusto table | where PreciseTimestamp > ago(5m) | summarize sum(Sum) by round([PreciseTimestamp], 5m), [provider]