Feature extraction analyzer
The feature extraction analyzer creates attributes out of event data based on regular expressions. Different
features can be specified in the
Please be aware that this analyzer does not extract ipv4, email-addresses and similar from all events, but only those that match the query_string.
This analyzer is helpful to built a list of
email_addresses in a sketch that are used in in
WEBHIST. To do that, run the analyzer to have the feature extracted. Check the results by running a query:
Those results now can be used in an aggregation to plot a table limited to that column.
Another way of extracting that information is via API, querying events that contain
email_address:* as a pandas dataframe, and work from there.
A feature extraction definition looks like this:
name: # Define either a query_string or query_dsl. query_string: * query_dsl: # Mandatory fields. attribute: store_as: re: # Optional fields. re_flags:  emojis:  tags:  create_view: False aggregate: False overwrite_store_as: True overwrite_and_merge_store_as: False store_type_list: False keep_multimatch: False
Each definition needs to define either a query_string or a query_dsl.
re_flags is a list of flags as strings from the re module. These include:
emojis are optional.
store_as defines the name of the attribute the feature is stored as.
create_view is an optional boolean that determines whether a view should be created if there are hits.
aggregate is an optional boolean that determines if we want to create an aggregation of the results and store it (ATM this does nothing, but once aggregations are supported it will).
overwrite_store_as is an optional boolean that determines if we want to overwrite the field
store_as if it already exists.
overwrite_and_merge_store_as is an optional boolean that determines if we want to overwrite the field
store_as and merge the existing values.
store_type_list is an optional boolean that determines if we want to store the extracted data in List type (default is text).
keep_multimatch is an optional boolean that determines if we want to store all matching results (default store first result).
The feature extraction works in the way that the query is run, and the regular expression is run against the attribute to extract a value. The first value extracted is then stored inside the "store_as" attribute. If there are emojis or tags defined they are also applied to that event. In the end, if a view is supposed to be created a view searching for the added tag is added (only if there are results).