Guides
Guides
v20151130
Guides
Guides

Query API

Suggest Edits

The connections application allows you to query against the various datasets that Thinknum compiles. These datasets allow you to examine various companies and their changing metrics.

Datasets List Endpoint

GET /datasets/ HTTP/1.1
Authorization: token 01234567890123456789
X-API-Version: 20151130

In response, you will get a JSON formatted block, containing the list of datasets. A list of items, each of which has the dataset id and its display_name.

A sample response follows:

{
    "datasets": [
        {
            "display_name": "Job Listings",
            "summary": "Understand current hiring plans and track changes daily.",
            "state": "ready",
            "blog_url": "https://blog.thinknum.com/job-listings/",
            "companies_count": 9962,
            "id": "job_listings"
        },
        ...
        {
            "display_name": "Google Play App Store Reviews",
            "summary": "Google Play App Store Reviews",
            "state": "ready",
            "blog_url": null,
            "companies_count": null,
            "id": "appstore_reviews_google_play"
        }
    ]
}

Datasets List for Ticker Endpoint

It's possible to limit the dataset list to a specific ticker by specific a "ticker" query parameter. For example, getting all datasets available for Apple Inc:

GET /datasets/?ticker=nasdaq:aapl HTTP/1.1
Authorization: token 01234567890123456789
X-API-Version: 20151130

A sample response follows:

{
    "datasets": [
        {
            "display_name": "Facebook App Active Users",
            "summary": "Track users logging into a website or app via their FB login credentials.",
            "state": "ready",
            "blog_url": "https://blog.thinknum.com/facebook-app-active-users/",
            "companies_count": 11596,
            "id": "social_facebook_app"
        },
        ...
        {
            "display_name": "Web Traffic",
            "summary": "Track unique visitors and their engagement to company websites.",
            "state": "ready",
            "blog_url": null,
            "companies_count": 15444,
            "id": "webtraffic"
        }
    ]
}

Ticker List Endpoint

To retrieve a list of tickers IDs that are available in a dataset, you must create a GET request with "query" parameter to:

GET /datasets/<DATASET_ID>/tickers/?query=apple HTTP/1.1

A sample response for the "Store Locations" dataset would look like this:

{
    "tickers": {},
    "queries": {
        "apple": [
            {
                "sector": "Technology",
                "display_name": "Apple",
                "url": "",
                "country": "USA",
                "industry": "Technology Hardware & Equipment",
                "entities": [
                    {
                        "display_name": "Apple Resellers",
                        "id": "Apple Resellers@nasdaq:aapl"
                    },
                    {
                        "display_name": "Apple",
                        "id": "Apple@nasdaq:aapl"
                    }
                ],
                "id": "nasdaq:aapl"
            },
          	...
            {
                "sector": "Consumer Services",
                "display_name": "DineEquity",
                "url": "",
                "country": "USA",
                "industry": "Travel & Leisure",
                "entities": [
                    {
                        "display_name": "Applebee's",
                        "id": "Applebee's@nyse:din"
                    },
                    {
                        "display_name": "IHOP",
                        "id": "IHOP@nyse:din"
                    }
                ],
                "id": "nyse:din"
            }
        ]
    }
}

How Query Endpoint Works

To retrieve a dataset for a specific company, you must create a POST request to:

POST /datasets/<DATASET_ID>/query/ HTTP/1.1
Authorization: token 01234567890123456789
X-API-Version: 20151130
Accept: application/json
Content-Type: application/json

{tickers: ["nasdaq:aapl"]}

curl -X POST "https://data.thinknum.com/datasets/<DATASET_ID>/query/" \
-H "Accept: application/json" \
-H "X-API-Version: 20151130" \
-H "Authorization: token 01234567890123456789" \
-d $'{"tickers": ["nasdaq:aapl"]}'

response = requests.post(
    url='https://data.thinknum.com/datasets/<DATASET_ID>/query/', 
    headers={
        'Authorization': 'token 01234567890123456789', 
        'X-API-Version': '20151130', 
        'Accept': 'application/json'
    }, 
    data=json.dumps({'tickers': ['nasdaq:aapl']})
)
results = json.loads(response.text)

The body should be serialized json contents.

A sample response follows:

{
    "state": "complete",
    "total": 49739,
    "id": "75a08474ca0992efc4acef3105abfd345a2d076e0f3938e8a94182f0221d33d4",
    "formats": [
        "application/vnd.thinknum.table+json",
        "application/vnd.thinknum.map+json"
    ]
}

The id parameter is QUERY_ID which is based on POST body and necessary for the following requests.

To check if query is completed, you must create a HEAD request to:

HEAD datasets/<DATASET_ID>/query/<QUERY_ID> HTTP/1.1
Authorization: token 01234567890123456789
X-API-Version: 20151130
Accept: application/vnd.thinknum.table+json
Content-Type: application/x-www-form-urlencoded; charset=utf-8

curl -I HEAD "https://data.thinknum.com/datasets/<DATASET_ID>/query/<QUERY_ID>" \
-H "Accept: application/vnd.thinknum.table+json" \
-H "X-API-Version: 20151130" \
-H "Authorization: token 01234567890123456789"

response = requests.head(
    url='https://data.thinknum.com/datasets/<DATASET_ID>/query/<QUERY_ID>', 
    headers={
        'Authorization': 'token 01234567890123456789', 
        'X-API-Version': '20151130', 
        'Accept': 'application/vnd.thinknum.table+json'
    }
)
results = dict(response.headers)

A sample response follows:

HTTP/1.1 200 OK
Server: nginx
Date: Fri, 30 Apr 2021 16:05:42 GMT
Content-Type: application/json
Connection: keep-alive
X-Truncated: false
X-Formats: application/vnd.thinknum.table+json, application/vnd.thinknum.map+json, text/csv, application/vnd.ms-excel, application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
X-Total: 49739
X-State: complete
Strict-Transport-Security: max-age=15768000

X-Truncated: The results will be truncated if it is over 500,000. It will be true or false
X-Formats: The available ACCEPT for the following GET endpoint.

  • application/vnd.thinknum.table+json: Table
  • text/csv: CSV
  • application/vnd.ms-excel: XLS
  • application/vnd.openxmlformats-officedocument.spreadsheetml.sheet: XLSX

X-Total: The number of returned rows
X-State: complete, running

To retrieve data, you must a GET request to:

GET /datasets/<DATASET_ID>/query/<QUERY_ID>?limit=10&start=1
Authorization: token 01234567890123456789
X-API-Version: 20151130
Accept: application/vnd.thinknum.table+json
Content-Type: application/json

curl -X GET "https://data.thinknum.com/datasets/<DATASET_ID>/query/<QUERY_ID>?limit=10&start=1" \
-H "Accept: application/vnd.thinknum.table+json" \
-H "Content-Type: application/json" \
-H "X-API-Version: 20151130" \
-H "Authorization: token 01234567890123456789"

response = requests.get(
    url='https://data.thinknum.com/datasets/<DATASET_ID>/query/<QUERY_ID>', 
    headers={
        'Authorization': 'token 01234567890123456789', 
        'X-API-Version': '20151130', 
        'Accept': 'application/vnd.thinknum.table+json'
    }
)
results = json.loads(response.text)

Optional parameters can be specified to start your request at a certain record or limit the amount of records. For example, the above request will get records 1-1000.

A sample response follows:

{
    "group_fields": [
        {
            "display_name": "Ticker Symbol",
            "format": "ticker",
            "metric": false,
            "id": "dataset__entity__entity_ticker__ticker__ticker",
            "length": 320,
            "summary": "The full ticker symbol, defined as the financial market of the company, followed by a colon (:), and then the identifier of the company. ex. NASDAQ:AAPL",
            "type": "string",
            "options": []
        },
        ...
        {
            "display_name": "GICS Industry",
            "format": null,
            "metric": false,
            "id": "dataset__entity__entity_ticker__ticker__market_industry",
            "length": 64,
            "summary": "The gics industry within the sector a ticker belongs to",
            "type": "string",
            "options": []
        }
    ],
    "total": 500000,
    "id": "8fe16c38475db71c6223ca7be36bf200cca5b1410face517b5f33c315f1a83d3",
    "count": 10,
    "rows": [
        [
            "nasdaq:aapl",
            "jobs.apple.com",
            "037833100",
            "us0378331005",
            4505809365993,
            "https://jobs.apple.com/en-us/details/200240406/sre-engineer-manufacturing-design-systems?team=SFTWR",
            "jobs.apple.com",
            "20210425",
            "SRE Engineer  - Manufacturing Design Systems",
            "https://jobs.apple.com/en-us/details/200240406/sre-engineer-manufacturing-design-systems?team=SFTWR",
            null,
            "Software and Services",
            null,
            null,
            null,
            null,
            "20210422",
            null,
            "We are looking for an SRE who can help lead the next generation of products we create. Our infrastructure team is responsible for architecting, building, and scaling a distributed system that enables Apple to manufacture every product. We manage hundreds of bare-metal servers and thousands of client machines across 10+ data centers. You should strive to make everyone that uses these systems life easier including devs, technical support, on location teams, and end users. This means automating everything from deployment workflow to CI/CD to monitoring and alerting systems.",
            "20210424T220423Z",
            "20210424T220423Z",
            "Technology",
            "Technology Hardware & Equipment"
        ],
        ...
        [
            "nasdaq:aapl",
            "jobs.apple.com",
            "037833100",
            "us0378331005",
            4505809367986,
            "https://jobs.apple.com/en-us/details/200228325/flex-supplier-quality-engineer?team=OPMFG",
            "jobs.apple.com",
            "20210425",
            "Flex Supplier Quality Engineer",
            "https://jobs.apple.com/en-us/details/200228325/flex-supplier-quality-engineer?team=OPMFG",
            null,
            "Operations and Supply Chain",
            "Hanoi",
            "",
            "",
            "VNM",
            "20210317",
            null,
            "The people here at Apple don't just build products - they build the kind of wonder that's revolutionized entire industries. It's the diversity of those people and their ideas that encourages the innovation that runs through everything we do, from amazing technology to industry-leading environmental efforts. Join Apple, and help us leave the world better than we found it.\n\nThis position is for a supplier quality engineer who will focus on sustaining Hot Bar assembly & flexible printed circuits (include product quality yield, LRB) management.",
            "20210424T220423Z",
            "20210424T220423Z",
            "Technology",
            "Technology Hardware & Equipment"
        ]
    ],
    "fields": [
        {
            "display_name": "Ticker Symbol",
            "format": "ticker",
            "metric": false,
            "id": "dataset__entity__entity_ticker__ticker__ticker",
            "length": 320,
            "summary": "The full ticker symbol, defined as the financial market of the company, followed by a colon (:), and then the identifier of the company. ex. NASDAQ:AAPL",
            "type": "string",
            "options": []
        },
        ...
        {
            "display_name": "GICS Industry",
            "format": null,
            "metric": false,
            "id": "dataset__entity__entity_ticker__ticker__market_industry",
            "length": 64,
            "summary": "The gics industry within the sector a ticker belongs to",
            "type": "string",
            "options": []
        }
    ],
    "sort_fields": [
        {
            "column": "as_of_date",
            "order": "desc"
        },
        {
            "column": "domain",
            "order": "asc"
        }
    ],
    "start": 1,
    "state": "complete",
    "limit": 10,
    "last_date_updated": "2021-04-24T22:04:23Z"
}

Ticker-less Queries

If you're interested in all data for a dataset, you can omit or specify an empty ticker list to avoid filtering to tickers.

POST /dataset/store/query/ HTTP/1.1
Authorization: token 01234567890123456789
X-API-Version: 20151130

{"tickers": []}

curl -X POST "https://data.thinknum.com/datasets/store/query/" \
-H "Accept: application/json" \
-H "X-API-Version: 20151130" \
-H "Authorization: token 01234567890123456789" \
-d $'{"tickers": []}'

response = requests.post(
    url='https://data.thinknum.com/datasets/store/query/', 
    headers={
        'Authorization': 'token 01234567890123456789', 
        'X-API-Version': '20151130', 
        'Accept': 'application/json'
    }, 
    data=json.dumps({'tickers': []})
)
results = json.loads(response.text)

A sample response follows:

{
    "state": "running",
    "total": 0,
    "id": "427b5c6390fdc0e579fb1f09eebd1b9bd31e40f21a7a01c2f40e83949f78c344",
    "formats": [
        "application/vnd.thinknum.table+json",
        "application/vnd.thinknum.map+json"
    ]
}

To check if query is completed:

HEAD datasets/store/query/427b5c6390fdc0e579fb1f09eebd1b9bd31e40f21a7a01c2f40e83949f78c344 HTTP/1.1
Authorization: token 01234567890123456789
X-API-Version: 20151130
Accept: application/vnd.thinknum.table+json
Content-Type: application/x-www-form-urlencoded; charset=utf-8

curl -I HEAD "https://data.thinknum.com/datasets/store/query/427b5c6390fdc0e579fb1f09eebd1b9bd31e40f21a7a01c2f40e83949f78c344" \
-H "Accept: application/vnd.thinknum.table+json" \
-H "X-API-Version: 20151130" \
-H "Authorization: token 01234567890123456789"

response = requests.head(
    url='https://data.thinknum.com/datasets/store/query/427b5c6390fdc0e579fb1f09eebd1b9bd31e40f21a7a01c2f40e83949f78c344', 
    headers={
        'Authorization': 'token 01234567890123456789', 
        'X-API-Version': '20151130', 
        'Accept': 'application/vnd.thinknum.table+json'
    }
)
results = dict(response.headers)

A sample response follows:

HTTP/1.1 200 OK
Server: nginx
Date: Wed, 28 Apr 2021 16:55:45 GMT
Content-Type: application/json
Connection: keep-alive
X-Truncated: true
X-Formats: application/vnd.thinknum.table+json, application/vnd.thinknum.map+json, text/csv
X-Total: 500000
X-State: complete
Strict-Transport-Security: max-age=15768000

To retrieve data:

GET /datasets/store/query/427b5c6390fdc0e579fb1f09eebd1b9bd31e40f21a7a01c2f40e83949f78c344
Authorization: token 01234567890123456789
X-API-Version: 20151130
Accept: application/vnd.thinknum.table+json
Content-Type: application/json

curl -X GET "https://data.thinknum.com/datasets/store/query/427b5c6390fdc0e579fb1f09eebd1b9bd31e40f21a7a01c2f40e83949f78c344?limit=10&start=1" \
-H "Accept: application/vnd.thinknum.table+json" \
-H "Content-Type: application/json" \
-H "X-API-Version: 20151130" \
-H "Authorization: token 01234567890123456789"

response = requests.get(
    url='https://data.thinknum.com/datasets/store/query/427b5c6390fdc0e579fb1f09eebd1b9bd31e40f21a7a01c2f40e83949f78c344', 
    headers={
        'Authorization': 'token 01234567890123456789', 
        'X-API-Version': '20151130', 
        'Accept': 'application/vnd.thinknum.table+json'
    }
)
results = json.loads(response.text)

A sample response follows:

{
    "group_fields": [
        {
            "display_name": "Ticker Symbol",
            "format": "ticker",
            "metric": false,
            "id": "dataset__entity__entity_ticker__ticker__ticker",
            "length": 320,
            "summary": "The full ticker symbol, defined as the financial market of the company, followed by a colon (:), and then the identifier of the company. ex. NASDAQ:AAPL",
            "type": "string",
            "options": []
        },
        ...
        {
            "display_name": "GICS Industry",
            "format": null,
            "metric": false,
            "id": "dataset__entity__entity_ticker__ticker__market_industry",
            "length": 64,
            "summary": "The gics industry within the sector a ticker belongs to",
            "type": "string",
            "options": []
        }
    ],
    "total": 500000,
    "id": "427b5c6390fdc0e579fb1f09eebd1b9bd31e40f21a7a01c2f40e83949f78c344",
    "count": 10,
    "rows": [
        [
            "private:simplemills",
            ...
            "Household Goods"
        ],
        ...
        [
            "private:simplemills",
            ...
            "Household Goods"
        ]
    ],
    "fields": [
        {
            "display_name": "Ticker Symbol",
            "format": "ticker",
            "metric": false,
            "id": "dataset__entity__entity_ticker__ticker__ticker",
            "length": 320,
            "summary": "The full ticker symbol, defined as the financial market of the company, followed by a colon (:), and then the identifier of the company. ex. NASDAQ:AAPL",
            "type": "string",
            "options": []
        },
        ...
        {
            "display_name": "GICS Industry",
            "format": null,
            "metric": false,
            "id": "dataset__entity__entity_ticker__ticker__market_industry",
            "length": 64,
            "summary": "The gics industry within the sector a ticker belongs to",
            "type": "string",
            "options": []
        }
    ],
    "sort_fields": [],
    "start": 1,
    "state": "complete",
    "limit": 10,
    "last_date_updated": "2021-04-28T02:44:53Z"
}

Updated over 3 years ago