Step 4: Applications of Functions

Connectors Transforms

If you want to write a Function to use as a Transform in Data Connectors, the Function must have a string => string signature, meaning it will take a single string argument and return a single string. This is because the Function will be invoked on each cell of the selected column or columns in your preview table and the value returned will replace the original cell value.

export function myCustomTransform(myInputString: string): string {
    /** TODO: write transform logic */
    return myOutputString;
}

Connectors Sources

You can also write a Function to serve as a data source for a Data Connector. When your Connector is run, the function will be invoked. Similar to Transforms, the Function must have a string => string signature, meaning it will take a single string argument and return a single string. However, when using a Function as a Connector source, the strings passed into and returned by the Function should be stringified JSON objects.

The string that should be passed in as a Function argument will be a stringified JSON object that has one key: pageToken. This is so that your Function can support API pagination. The first time the Function is invoked by your Connector, the pageToken value will be blank, but if your Function needs to be invoked more than once to fetch multiple pages of data, a pageToken value can be passed back into the Function. More on this below.

Once a Function invocation is complete, it should return a stringified JSON object that has two keys: data and nextPageToken. data is the JSON data returned by the Function — it must be valid JSON structure in order to be processed by the Connector. nextPageToken is an optional value that should be set if there are more pages of data to be fetched — the value for nextPageToken is passed back to your Function in the pageToken key as described above. If no nextPageToken is returned, the Connector will assume there are no more pages of data to fetch and will not invoke your Function again.

export async function myCustomDataSource(inputString: string): string {
    const inputJson = JSON.parse(inputString)
    const pageToken = inputJson["pageToken"]
    
    let requestUrl = 'https://myrequesturl.com'
    if (pageToken != null) {
        requestUrl = `${requestUrl}&pagination_token=${pageToken}`
    }

    const response = await fetch(requestUrl).then(response => response.json())
    const nextPageToken = response["meta"]["next_token"]
    const outputString = JSON.stringify({"data": response, "nextPageToken": nextPageToken})

    return outputString;
}