Release
Sophie standalone versions 3.4.x - 3.7.x
Instructions
When Sophie ingests log data, it maps the data into three mandatory fields, affecting how it is parsed, analyzed, and correlated.
The fields are Source-Types, Services, and Applications.
Validate
To validate that your logs are structured optimally, navigate to the Kibana Log Analytics screen to see the final result of your logs after structuring.
Alternatively, if you already know the Source Type name, you can also check your log structure (and it's properties) by navigating to the source type screen.
Prepare
Best practices when adding your data, is to tag these properties in the configuration form prior to them reaching Sophie.
You can manually map them as well. See KB0821207 which includes steps to assist you with this process.
Source Types
A Source-Type is a mandatory field that defines how Sophie parses and measures log data.
Each data input can have up to any number of Source-Types, based on the variety of its logs’ formats.
Source-Types are mapped separately than Applications/Services and are not relational to each other as a domain-co-domain, allowing for:
Every Application to have any number of Source-Types.
Every Service to have any number of Source-Types.
Services
In the context of Sophie’s data-ingestion, a Service is a mandatory field used for indexing and analyzing the data, based on the software generating the logs.
A Service doesn’t affect how Sophie parses data, instead, it gives Sophie a technical context on each log’s source.
Services represent the small technical components or business processes in Sophie’s data mapping, thus several of them are often clustered together under one Application.
Applications
In the context of Sophie’s data-ingestion, an Application is a mandatory field used for indexing and analyzing the data. Each organization may configure Applications differently resulting in a single log file, generated by the same 3rd-party software in two different organizations, indexed and analyzed differently.
An Application doesn’t affect how Sophie parses data, instead, it gives Sophie a business-context on each log’s source, allowing Sophie to perform another layer of correlation.
Applications represent larger or full business processes in Sophie’s data mapping and consists of either one or more Services.
Every Application can have up to any number of Services.
Different Applications can be composed of the same Services.
Data-Ingestion Flow
The following figures visualizes Sophie’s data-ingestion flow:
