Data Fetching| Hitchhikers Platform

Overview

Most websites will want to fetch data and then use that data to populate the content on a webpage. Generally, you will want to use the Knowledge Graph to store and manage this data but there are other approaches as well. This document will walk through the different ways you can manage data and pull it into your templates.


From Knowledge Graph

Data from the Knowledge Graph can be directly managed through the Knowledge Graph UI or the connectors framework to sync content from external data sources.


Page per Entity

The most common scenario is to create a page for each entity in your graph. This could be for products, locations, help articles, or anything else. The KG schema is completely flexible and you can use the Knowledge Graph as a full-blown CMS.

If you want to create a page per entity then you will want to create a template that is hooked up to a “stream”. A stream is a filtered set of entities in your Knowledge Graph. As the name implies this data source is streamed to your site and any time you make a change in the graph, your site will automatically re-generate any impacted pages.


Global Data

Another scenario is when you want to use data from the Knowledge Graph across all your pages. This is useful for data in headers and footers that applies across all screen. We call this Global Data. Generally, you will create a single Site entity in your Graph and then have that automatically stream to every single template on your site.


From other APIs

Sometimes you will want to pull in content from APIs outside the Knowledge Graph. We generally recommend using a connector to pull data directly into the Knowledge Grap, but you can also pull it directly into your front end.. You can either pull in content at Generation Time or at Page Load Time.


Generation Time

Each page will only generate one time and will only re-generate when the entity that is powering that page changes. If you want to fetch data from an external API at generation time you should use the transformProps function discussed here in the templates doc.


Page Load Time

You can also hit third-party APIs at the time of page load. This is done client-side using standard react data fetching approaches. The simplest way to do this is to hit an API in a useEffect hook. See below. Note that this will run after the user sees the page so you will want to show a loading state.

import React, { useState, useEffect } from 'react';
import axios from 'axios';

function App() {
  const [data, setData] = useState({ hits: [] });

  useEffect(async () => {
    const result = await axios(
      'https://data.fetching.api',
    );

    setData(result.data);
  });

  return (
    <ul>
      {data.hits.map(item => (
        <li key={item.objectID}>
          <a href={item.url}>{item.title}</a>
        </li>
      ))}
    </ul>
  );
}

export default App;

This function will run every time the page loads. There are tons of third-party packages that can also help do this more easily. React-query is a great choice we recommend.

Page Load vs. Generation

Page load time is critical for SEO and user experience. The more that you can offload to static generation the better. If the API will not change over time we recommend using transformProps since that function will NOT impact page load time. Fetching data client-side is a good approach when the data changes super frequently AND it’s not crucial to the page (usually below the fold).

light bulb
Gotcha Alert
Neither of these approaches will cause pages to re-generate. If you want to regenerate a page each time data changes you should put that data in the Knowledge Graph.