Does NTerminal normalize incoming data across datasets?
All data is cleansed and normalized according to our data models at the first level of NTerminal’s Content Delivery Chain before being forwarded to other components. All currency symbols, tickers, trading venues, organization names, and other named entities are standardized to make it easier to cross-reference datasets and extract intelligence later.
Does NTerminal clean incoming data?
Following best practices in the IT security industry, we use data source redundancy to deal with erroneus and fraudulent data entering our system. This allows us to not only filter outliers, but also alert clients of suspicious activity that might affect them or their competitors who rely on a single source of “truth”. To detect important discrepancies in the technological layer of digital asset markets, we often collect the same information from multiple sources and apply 3-sigma control limits.
For example, NTerminal spot pricing data on Coinbase currently comes from 7 different sources. If any of those sources go offline or start to provide significantly different information, that source will be dropped from further calculations. However, all collected information is saved and outliers will trigger quality alerts that are available to our engineers and our customers.
How reliable is NTerminal infrastructure?
NTerminal’s Content Delivery Chain (CDC) is built with a microservice architecture. This allows for redundant modules which can be rapidly deployed and modified. Splunk, which NTerminal builds on top of, is extensively used for compliance use cases and has built-in functionality that allows NTerminal to replicate data across our distributed infrastructure and automatically recover after node failures or data corruption.
How does NTerminal figure out which sources are good and bad?
We have expertise in data collection and analysis. Our co-founder did it for INTERPOL. We also have over 5 years of experience in the crypto space. All of the data we gather is public information. We work to source, organize, and parse all that data. While we internally prioritize the data sources that we think matter, we are happy to integrate client requests, and have many redundant sources. Customers can buy finance data or linguistics data regarding any of the topics that we source, and can always request new data streams for integration.
I don’t get it, can you give an example?
NTerminal can, for example, examine agents across spaces through our agent based modeling project. NTerminal can find who is subscribing to a specific developer on Reddit and find out who he/she influences. Let’s imagine there is a developer that says Monero is doomed. We know the developer is legitimate and we can see that people are re-tweeting him, then blog posts are written about his opinion that Monero will fail, and then it hits the Wall Street Journal. We will be able to predict the price of Monero in advance as the story advances. NTerminal also has investigation tools. NTerminal can find the breadcrumbs all the way to the developer who originally posted on Reddit.
How does the data update?
It depends on how we get the data and where it is, but all updates are real-time.
Where does NTerminal get data from? Does NTerminal have a list of data sources?
It doesn’t make sense to provide a static list of data sources because we rely on the dynamic information discovery functionality of NTerminal, as well as leverage 3rd party aggregators of data. For example, we monitor Google News for relevant keywords, but do not control how Google decides to classify news media on any given day. In addition to this, we follow links posted by people and organizations we identify as belonging to a community we monitor (individuals from: blockchain, technology, security blockchain mining, government, finance, central banks, trading venues, science, darknet, media, etc.)
Can NTerminal customize the levels of information delivery?
Yes. Customers can add their own search terms to the data we have already parsed, or identify new sources of data for us to parse.
What if I’m a hedge fund and I don’t want other people being able to see what I’m searching for?
No one will be able to see the data you upload or your search history. You can share it with selected users from your company if you choose to, or keep it completely private. In addition to this, we built NTerminal following “infrastrustructure as code” principles, which makes it possible to spin up our infrastructure as an on-premise solution or in a private cloud in a matter of hours.
Do customers have visibility into NTerminal algorithms? Do they have visibility into any of the methodology for analyzing or scraping the datasets? What type of customization will they be able to apply?
By using drill-down functionality, NTerminal users have full access to the logic behind every graph, panel, and dashboard, as well as the raw data streams and derived data that were used to build them.
Most of our code is open source, with a thin layer of proprietary configuration data. How we process the information is private. Think of a chef who proudly shows off his kitchen, his tools, and his ingredients, but keeps the exact recipe to himself. That said, we enable third party companies (for example another company that sells data) to use our data or build on top of it for their own purposes. We can put other companies in the middle of our infrastructure. They can get our raw data and then they can process it, get the result, and then feed it back into our system, or their own. If someone finds that their data is reliable or useful, they can make money selling their data through our system.
How easily can structured data be pushed into NTerminal?
Very easily. Only last-mile changes are required. Splunk makes this very easy with CSV imports, lookup tables, GUI field extractions, etc.
How easily can structured data be exported from system?
Very easily. Our engineers build custom APIs to a client’s specification for historical and real-time delivery. Additionally, Splunk has this functionality built in. CSV, Excel, PDF, it’s all there. Real-time export is also possible. Customers can forward data from their on-premise Splunk server using syslog or ask us to forward them data in whatever structured format they are ready to accept.
What are the realm of possibilities for new data integration? Does Inca have to do it or can users do it themselves? What hurdles?
We can integrate anything that can be obtained manually, so long as we know what users want. If it is subscription based, we can pay for it and obtain it. How quickly we can integrate depends on how well structured the data is and how many data formats are in the same source, but it usually takes anywhere from 15 min to a day.
There are no hurdles to NTerminal data integration. If a client can manually access it, we can provide you with a real-time stream in NTerminal. Users can integrate static data into our NTerminal online platform themselves. Moreover, users who are running their own version of Splunk (their own on premise solution) can input any data streams they want without going through us.