An exploration of our API Explorer

There’s this thing in software development called a black box. This is when data goes in one end of the box, the box does something to the data, and then something different comes out of the other end.

We don’t like black boxes at SyncHub - they obfuscate the service that we are offering, and make it harder for people to understand exactly what is happening to their data. We figure most of our customers are like us, and like to know how things work - not just trust the black box and take the output for granted.

So today we are introducing our API Explorer. This is the tool that the SyncHub engine uses to query cloud services and convert your data into a relational format suitable to query.

A quick reminder about SyncHub and what it does…

Getting your data out of the cloud and into somewhere you can use it is what SyncHub is all about. The engines driving this process are our connectors, the custom software we build specifically to talk to your APIs. These connectors extract your data, organize it into a usable structure, and then place it into your relational data warehouse - ready for you to play with to your heart’s content.

This all happens automatically and continuously, and for most of our users that’s where SyncHub begins and ends - they plug in their credentials, set their sync periods and let the SyncHub magic work. We make it look easy, but there is a lot going on behind the scenes - but that’s what you pay us for, right?. We put a lot of effort into making this process just work.

Introducing…the API Explorer

The API explorer is an interface that lets you run queries against any of our supported API’s, using our connector engine. It provides highly detailed & transparent insights into how your cloud service API works, and what precisely SyncHub is doing to move your data into your warehouse.

Common uses of the API Explorer include:

• maybe we need a specific record, or can’t quite see how the values from a service are mapping to the values in our warehouse

• perhaps we are exploring an API for a new data point

• are you getting values in your reporting tool that don’t match what you see on the service? Pull that item directly off the API and see what it’s returning

• don’t understand where some of the columns in your warehouse are coming from? Check the raw data and compare it to our transformed object

• want to find out what other data you can retrieve from your service? Whip up a custom query and see

• or, perhaps you are like us and simply believe that the data you load into a cloud service is yours and should be easily accessible and available to you

How do I access the API Explorer?

To get to the API Explorer in SyncHub, simply visit your Connection Dashboard and select from the Options menu. SyncHub will automatically load the API Explorer for that connection with your credentials in place, ready for you to query immediately:

The Data Model

Before you begin, please note the Data Model which is displayed on the left side of the API Explorer. As you select from the list of API methods available, this data model provides a helpful list of all the properties this endpoint deals with, split into the properties that are directly placed in your store and the ones our system calculates. You are also given other helpful information such as foreign keys, any known issues with this endpoint and a link to the relevant API documentation.

Querying the SyncHub endpoints

On the left of the API Explorer, you’ll see a list of our built in methods. These are the same ones actually used by SyncHub to retrieve your data, providing complete transparency into how our engine operates.

Note: all of these methods are read-only so you can’t accidentally mess up your cloud data. It’s important to note however that the API Explorer is using a live connection to the service API, so requests count against your API throttling limit. This means it’s possible to accidentally use up all or a large chunk of the limit with a complex query.

Also: the API Explorer never loads any of the data it retrieves into your warehouse. It is purely for examining the data that is returned from an API.

To execute a method, simply select the endpoint that we want and make a call. Say we want to get all the Customers from our Unleashed account:

1. We first select the GetCustomers method as this will return all customers modified between the selected dates

2. Enter the dates within which you wish to search (here we leave the date filters blank because we want everything)

3. Finally, hit Execute and the request is fired off. You can watch in real time as the API is called and returns its payload

As you can see, once the API call completes, we present the result on the right-hand side of the API Explorer. Note that the data is presented differently in three different tabs:

• Materialized

• Explore

• Raw

Let’s examine these in a little more detail…

Materialized

The first tab shown here is the “Materialized” data. This means it contains the transformed information that would end up in your data store if SyncHub had run this request. Most specifically, it includes values for RemoteID/Deleted/WhenModified/WhenCreated - critical information used by SyncHub to monitor and sync changes to your data.

Explore

You can see the materialized data in a friendlier format by opening the “Explore” tab. This loads our interactive model of the materialized info and displays a much cleaner view. It also allows you to collapse and expand nested parts of the payload, or drill down into referenced entities simply by clicking on a known foreign key value

Raw

To see the raw data exactly as it comes down from the API select the “Raw” tab. This display lists the individual API calls made by your request and the data returned by them. As well as letting you see precisely what is returned by an API call this can allow you to get a sense of how many calls are required to get the data you want. If you ever wonder why you keep hitting your throttling limit, playing around with some high-use endpoints in the explorer will make it obvious.

Running custom queries

In addition to running the built-in SyncHub queries, the API Explorer offers the ability to create a completely custom query. This is especially useful when exploring your cloud service for additional endpoints that SyncHub is not currently bringing down into your warehouse. In fact, our developers use this tool all the time when building out our product.

You can access the custom queries by selecting Other from the endpoint menu on the left:

As you can see, the custom queries provide complete fidelity over how your query is constructed and executed. Those of you familiar with HTTP services will recognize all the features below, but it’s worth quickly going over them again here.

Request type

Because SyncHub concentrates on reading data, we currently offer only GET and POST methods now. As to which one you should use, well that depends on the API you are connecting to. If one doesn’t work, try the other.

Encoding

POST methods allow you to submit your data in a variety of ways. All the APIs we have encountered thus far are either form/multipart or JSON encoding. If you select the former, we provide a nice UI to let you set your key/value pairs (e.g. “search” = “Larry Phillips”). If the latter, simply type your JSON blob into the “Request body” textbox.

Authentication

Ninety-nine times out of a hundred you will need to be authenticated in order to make calls against the API, so we recommend you leave this on. In fact, the only times we don’t use this is when we are testing the actual authentication/login methods themselves.

That’s pretty cool

We think so too. We use this tool ourselves for troubleshooting and familiarization and have designed it with our own needs in mind. But we also believe strongly in the idea of Data Accessibility. We believe that the data you put into these services - including our own - is yours. It should be easy for you to get it and easy to see how it is handled. The API Explorer shows you how SyncHub actually gets your data and what it does with it. It lets you use the connectors we’ve built to get this data for yourself if you want it, and we think that’s important.

We hope you use the API Explorer yourselves, and gain a better insight into how your data is stored and handled. But even if you never use this feature, the fact that it is there at all is emblematic of what we want SyncHub to be about.