Data Provider

Simple but very powerful client, it was designed to help you consume data easily.

installyarn add @clayui/data-provider
versionNPM Version
useimport DataProvider, {useResource} from '@clayui/data-provider';

Introduction

ClayDataProvider gives functionality of data caching, attempts, polling, network status and avoiding the thundering herd problem. It is simple and powerful because:

  • Easy adoption , you can incrementally use in your application and both useResource hook and ClayDataProvider component and have all the functionality available.
  • Simple to start , use the basics you already know or take advantage of the full set of features to get the most out of it.
  • Built for data to reflect what users are doing in your application , it works perfectly for cases where data changes according to user interaction.
  • Extensible , enjoy the single cache in only one source of truth and save data between navigations to be used in future interactions.
  • Suspense and ErrorBoundary do incremental adoption with <React.Suspense /> and <ErrorBoundary /> to the new React patterns.

Getting started

To consume data, you can work with two different ways in React, using the <ClayDataProvider /> component or useResource hook. We recommend that you use useResource for cases where your component has more logic to handle data, so it decreases the complexity and eliminates logic within JSX, use <ClayDataProvider /> for simpler cases that do not have so much logic involved in the data or when you are not familiar with hooks.

<ClayDataProvider link="https://rickandmortyapi.com/api/character">
	{({data, error, loading, refetch}) => {}}
</ClayDataProvider>

useResource hook

The vast majority of APIs are the same between useResource and <ClayDataProvider />, the difference is that there is no notifyOnNetworkStatusChange API in useResource, you control them via the OnNetworkStatusChange parameter when you need it.

const App = () => {
	const {resource} = useResource({
		link: 'https://rickandmortyapi.com/api/character/',
	});

	return null;
};

Features

Retry

Make attempts on a request several times when a network or server error occurs.

fetchRetry is easy to set up and is enabled by default with the jitter setting for delays between attempts by default.

Warning The values ​​contained in the code below are the default value.

const App = () => {
	const {resource} = useResource({
		link: 'https://rickandmortyapi.com/api/character/',
		fetchRetry: {
			attempts: 5,
			delay: {
				initial: 300,
				jitter: true,
			},
		},
	});

	return null;
};

Network Status

The DataProvider provides network status information for you if you want to create customizations in those statuses. If you are using <ClayDataProvider /> you can enable this information by activating the notifyOnNetworkStatusChangeAPI prop, once activated it will cause new renderings each time the network status changes.

const App = () => (
	<ClayDataProvider
		link="https://rickandmortyapi.com/api/character"
		notifyOnNetworkStatusChange
	>
		{({data, error, loading, refetch, networkStatus}) => {}}
	</ClayDataProvider>
);

Using network status with hooks is another option, it does not provide an abstraction for loading, error and networkStatus and all information is collected through the onNetworkStatusChange callback.

  • loading is equivalent to networkStatus < 4
  • error is equivalent to status === 5
const App = () => {
	const [state, setState] = useState(() => ({
		error: false,
		loading: false,
		networkStatus: 4,
	}));
	const {resource} = useResource({
		link: 'https://rickandmortyapi.com/api/character/',
		onNetworkStatusChange: (status) =>
			setState({
				error: status === 5,
				loading: status < 4,
				networkStatus: status,
			}),
	});

	return null;
};

Variables change

variables is an API for GET requests that help satisfy whether your cache will be retrieved from storage or not, this can be useful for cases where your data is formed by user interactions such as Autocomplete, you can still set a delay on the fetchDelay prop to ensure that your requests are not called every time a change of input value occurs, for example.

const App = () => {
	const [value, setValue] = useState('Rick');
	const {resource} = useResource({
		link: 'https://rickandmortyapi.com/api/character/',
		fetchDelay: 300,
		variables: {name: value},
	});

	return null;
};

Caching data

You can cache your requests so that in new user interactions a new request is no longer necessary, by default the cache is deactivated.

The cache is guided by a policy, use the fetchPolicy prop to enable and configure the cache according to your use case.

Warning The cache is governed by the algorithm least recently used ( LRU ), you can set the amount of data that will be stored using the storageMaxSize API. Each new query is equivalent to 1 size.

Warning When suspense is enabled the hook automatically changes the fetchPolicy to FetchPolicy.CacheAndNetwork if it is set to FetchPolicy.NoCache so it works properly.

Infinite loading

Using the useResource hook also supports paginated data, which is common in APIs to avoid large amounts of data being trafficked in just one request and decrease response time. To get this behavior you need to return the cursor value with the next page link in the fetch property.

Data is automatically aggregated as new requests are being made when the loadMore callback is called.

Info Calling the refetch callback as an attempt to refresh the data will not work since the data is already aggregated.

const {loadMore, resource} = useResource({
	fetch: async (link: string) => {
		const result = await fetch(link);
		const json = await result.json();

		return {
			cursor: json.info.next,
			items: json.results,
		};
	},
	link: 'https://rickandmortyapi.com/api/character',
	variables: {limit: 10},
});

Suspense and ErrorBoundary

The useResource hook also supports integration with <React.Suspense /> which allows you to “suspend” the component while a request is in progress and add a component in fallback and can in some scenarios parallelize the requests. Read more about using Suspense to load data and the benefits it can bring to your application.

const Menu = () => {
	const {resource} = useResource({
		link: 'https://rickandmortyapi.com/api/character',

		// Enable Suspense integration
		suspense: true,
		variables: {limit: 10},
	});

	return (
		...
	);
};

const Root = () => (
	<React.Suspense fallback={<LoadingIndicator />}>
		<Menu />
	</React.Suspense>
);

If a network error happens you can also catch the error in render time using <ErrorBoundary /> and render some component as fallback when this happens and add the possibility for the component to try to recover.

const Root = () => (
	<ErrorBoundary>
		<React.Suspense fallback={<LoadingIndicator />}>
			<Menu />
		</React.Suspense>
	</ErrorBoundary>
);

Data Fetching

const {resource} = useResource({fetch, link});

This is an API that replaces the link behavior of receiving an async function, this did not allow us to validate the cache correctly because we do not have the URL view. This API is more friendly and has a unique responsibility, you may be able to pass an async function that accepts the link and options, and return the data. useResource will use its async function instead of the fetch default.

Fetch

import fetch from 'unfetch';

const App = () => {
	const {resource} = useResource({fetch, link: 'https://clay.dev'});
	// ...
};

Sortable

const {resource, sort, sortChange} = useResource({
	fetch: (link, init, sort) => {
		const url = new URL(link);

		if (sort) {
			url.searchParams.append('column', sort.column);
			url.searchParams.append('direction', sort.direction);
		}

		return fetch(url, init);
	},
	link: 'https://clay.dev',
});

Advanced

Avoiding thundering herd problem

Starting with delay.initial, the delay of each subsequent retry is increased exponentially, meaning it’s multiplied by 2 each time. For example, if delay.initial is 100, additional retries will occur after delays of 200, 400, 800, etc.

With the jitter option enabled, delays are randomized anywhere between 0ms (instant), and 2x the configured delay. This way you get the same result on average, but with random delays.

These two features combined help alleviate the thundering herd problem, by distributing load during major outages. Without these strategies, when your server comes back up it will be hit by all of your clients at once, possibly causing it to go down again.

Warning The implementation of this was based on the apollo-link-retry plugin for React Apollo.

Caching data at root level

The DataProvider can be used on small components that need some data and if it is very reused by the application in other pages, it does not make sense to consult this data every time the user interacts with it in other parts of your application, you can take advantage of the root level cache, ensuring that the next user interactions in the component are with data in the cache, even if it is on other pages.

Warning The use of the storage property has been deprecated since v3.67.0 in favor of declaring the storageMaxSize=100 component in the application root to control cache state and other internal details.

const App = () => {
	const storageContext = useContext(Store);

	return (
		<ClayDataProvider
			link="https://rickandmortyapi.com/api/character"
			fetchPolicy="cache-first"
			storage={storageContext}
		>
			{({data, error, loading, refetch}) => {}}
		</ClayDataProvider>
	);
};

API Reference

DataProvider

({ children, notifyOnNetworkStatusChange, ...otherProps }: IProps) => JSX.Element
Parameters
Properties

children *

(props: ChildrenProps) => React.ReactElement

It uses a render props pattern made popular by libraries like React Motion and React Router.

Children as a function is required for the DataProvider to pass the props with data information, network status, refetch method and others. If this is an impediment try using the useResource hook.

props *

ChildrenProps

data *

any

error *

string | boolean

loading *

boolean

networkStatus

NetworkStatus | undefined

refetch *

Function

notifyOnNetworkStatusChange

boolean | undefined

Set to true means that network status information will be passed via renders props and will also cause new renderings as networkStatus changes, when false rendering does not happen again.

fetch

(<T = unknown>(link: string, init?: RequestInit | undefined, sort?: Sorting | undefined) => Promise<Response> | Promise<FetchCursor<T>> | Promise<T>) | undefined

A Promise returning function to fetch your data, this replaces the use of fetch by default.

fetchDelay

number | undefined

This API is used in conjunction with variables API, if it is always changing its value set a debounce time to make a new request.

Set a value in ms.

fetchOptions

RequestInit | undefined

Options passed to request configuration.

fetchPolicy

FetchPolicy | undefined

Fetch policy is an option that allows you to specify how you want your component to interact with the cache.

(cache-first) Whenever a new request occurs, the data provider will first look at the cache and return it if it satisfies the data, otherwise it will perform the request. (no-cache) It will always make a new request and return the result and the cache is deactivated. When using this with suspense enabled the policy changes to cache-and-network to make it work better, you can still change it to cache-first as well. (cache-and-network) This case is specific to when you want your users to have a quick response, when a new request happens, it will first go to the cache and if it exists it will return and regardless if it is in the cache it will make a new request on the network.

The data provider takes only the cached data when they meet the requirements, the variables and URL are what they define when the cache satisfies their request. Be careful if your request is needed to cache to avoid problems.

fetchRetry

FetchRetry | undefined

Define the strategies for attempting new requests when it fails.

fetchTimeout

number | undefined

Set a request timeout in ms, if it reaches this limit it will go through the retry rules and if it still persists, it will set the networkStatus to 4 (Error).

link *

string | LinkFunction

Set the URL to where the data provider will have to make a request, by default the request is solved with json, if it does not cover your use case you can also pass a function by returning a Promise. We do not recommend that you use a function to do so, you will lose some benefits of the data provider, always try to avoid.

(!) The behavior of the link accepting a function has been deprecated in favor of the fetcher API.

pollInterval

number | undefined

The interval is set in milliseconds, setting the value to zero will disable polling.

storageMaxSize

number | undefined

Set the amount of items that can be cached, set to zero will be treated as infinite, be aware to set an ideal size to offer a positive experience for your user but not use a large amount of memory.

suspense

boolean | undefined

Flag to enable useResource integration with suspense.

variables

Variables | undefined

Variables are analyzed and converted to be passed as parameters to the query of a GET request, for example.

storage

Record<string, any> | undefined

Reference your storage provider, like Context, Store Object... Whenever a new request happens the data provider will look at storage, respecting the fetch policy.

If you have a context serving as your store in your application independent of where the user is interacting the data provider can retrieve the data from the cache again if it is satisfied. The data is removed in LRU (least recently used) order.

Returns
Element

useResource

({ fetch: fetcher, fetchDelay, fetchOptions, fetchPolicy, fetchRetry, fetchTimeout, link, onNetworkStatusChange, pollInterval, storageMaxSize, suspense, variables, }: Props) => { loadMore: () => Promise<any> | null; refetch: () => void; resource: any; sort: Sorting | null; sortChange: (sort: Sorting | null) => void; }
Parameters

*

Props= {"fetchDelay":300,"fetchPolicy":"no-cache","fetchRetry":{},"fetchTimeout":6000,"onNetworkStatusChange":"() => {}","pollInterval":0,"suspense":false,"variables":null}

fetch

(<T = unknown>(link: string, init?: RequestInit | undefined, sort?: Sorting | undefined) => Promise<Response> | Promise<FetchCursor<T>> | Promise<T>) | undefined

A Promise returning function to fetch your data, this replaces the use of fetch by default.

fetchDelay

number | undefined= 300

This API is used in conjunction with variables API, if it is always changing its value set a debounce time to make a new request.

Set a value in ms.

fetchOptions

RequestInit | undefined

Options passed to request configuration.

fetchPolicy

FetchPolicy | undefined= "no-cache"

Fetch policy is an option that allows you to specify how you want your component to interact with the cache.

(cache-first) Whenever a new request occurs, the data provider will first look at the cache and return it if it satisfies the data, otherwise it will perform the request. (no-cache) It will always make a new request and return the result and the cache is deactivated. When using this with suspense enabled the policy changes to cache-and-network to make it work better, you can still change it to cache-first as well. (cache-and-network) This case is specific to when you want your users to have a quick response, when a new request happens, it will first go to the cache and if it exists it will return and regardless if it is in the cache it will make a new request on the network.

The data provider takes only the cached data when they meet the requirements, the variables and URL are what they define when the cache satisfies their request. Be careful if your request is needed to cache to avoid problems.

fetchRetry

FetchRetry | undefined= {}

Define the strategies for attempting new requests when it fails.

fetchTimeout

number | undefined= 6000

Set a request timeout in ms, if it reaches this limit it will go through the retry rules and if it still persists, it will set the networkStatus to 4 (Error).

link *

string | LinkFunction

Set the URL to where the data provider will have to make a request, by default the request is solved with json, if it does not cover your use case you can also pass a function by returning a Promise. We do not recommend that you use a function to do so, you will lose some benefits of the data provider, always try to avoid.

(!) The behavior of the link accepting a function has been deprecated in favor of the fetcher API.

onNetworkStatusChange

((status: NetworkStatus) => void) | undefined= "() => {}"

Callback is called when the network status is changed.

pollInterval

number | undefined

The interval is set in milliseconds, setting the value to zero will disable polling.

storage

Record<string, any> | undefined

Reference your storage provider, like Context, Store Object... Whenever a new request happens the data provider will look at storage, respecting the fetch policy.

If you have a context serving as your store in your application independent of where the user is interacting the data provider can retrieve the data from the cache again if it is satisfied. The data is removed in LRU (least recently used) order.

storageMaxSize

number | undefined

Set the amount of items that can be cached, set to zero will be treated as infinite, be aware to set an ideal size to offer a positive experience for your user but not use a large amount of memory.

suspense

boolean | undefined

Flag to enable useResource integration with suspense.

variables

Variables | undefined

Variables are analyzed and converted to be passed as parameters to the query of a GET request, for example.

Returns
{ loadMore: () => Promise<any> | null; refetch: () => void; resource: any; sort: Sorting | null; sortChange: (sort: Sorting | null) => void; }