Data Discovery

Due to the flexible nature of NTerminal, it is difficult for us to document all of the available data for each client. Requested sources are added quickly, and field names may differ based on a client’s needs. Rather than refering to static documentation of availible data types and their schemas, try using the following to help find what your looking for directly within your splunk instance. Additionally, it is important to note that customers who consume datasets with programmatic access in addtion to using using splunk may request different data streams (and/or data models) on their web platform than what they consume via API.

Important Field Names

Events will have differing field names and data models based on the type of data; some fields, however, are included in all events within Splunk that are used to categorize the event.

  • index - Every event has an index field which classifys what general category of data the event is. Example: Use index=financial to filter for financial data types, like trades or orders.
  • sourcetype - The sourcetype field gives a more specific classification of the data type for an event. Example: Within the financial index, use sourcetype=ohlcv to filter for trade data; “ohlcv” stands for “open, high, low, close, volume” which are all fields within the events for this sourcetype.
  • source - Each event also has a source field which will tell you where the event came from. Example: source=cdc tells you that this event is coming from our “content delivery chain”. You can use this field to investigate cases when sources show two different results.

Example Searches and Common SPL Logic

Bitcoin typical price for USD/BTC trades on Coinbase over time:

index=financial sourcetype=ohlcv symbol=BTC base=USD market_venue=COINBASE
| eval ohlc = ((open+high+low+close)/4) 
| timechart avg(ohlc) as price

Number of Zcash transactions during a selected time period

index=zcash source=cdc sourcetype=transaction information_source.id=zcash-node
| stats dc(content.hash)

Number of open issues and pull requests by repository

index=info sourcetype=github_orgs symbol=ETH
| stats latest(open_issues_count) as "Open Issues" latest(open_pull_requests_count) as "Open Pull Requests" by full_name

Basic spread & percent spread calculation

| eval ask_price=if(type="ask",price,null), bid_price=if(type="bid",price,null) 
| stats last(bid_price) as bid last(ask_price) as ask by market_venue,base
| eval mipoint=(ask+bid)/2, spread=ask-bid, percent_spread=spread/mipoint

Custom NTerminal Functions

We create custom search macros for commonly used functions for some clients. By navigating to “Advanced Search” within settings, users can see the names, definitions, and possible arguments for all available macros. Please note: users must add a ` to either side of the following searches.

  • nlp_sentiment_asset(symbol) - categorizes nlp events into “negative” “neutral” or “positive”
  • relative_spread(symbol,market_venue,base) - calculates the relative spread volatility for an asset
  • address_activity(blockchain, address) - returns the number of transactions and total volume for address(es) on a selected blockchain
  • github_stats(symbol) - returns github statistics by project for a selected asset

Example:

`address_activity(bitcoin,1Nh7uHdvY6fNwtQtM1G5EZAFPLC33B59rB OR 1Mk1oumdtDpCmCwCovSVjWgtiSToXvgELw)`

Use our Blockchain Connector to return information about an address or transaction. See examples:

| getaddresstags address="1Mk1oumdtDpCmCwCovSVjWgtiSToXvgELw"

index=ethereum sourcetype=transaction 
| head 1000
| addaddresstags inputfield=content.to outputfield=tags_out
| sort tags_out
| table content.hash content.to tags_out