The Fetch API

JavaScript Cannot Fetch Data From Other URLs

JavaScript cannot get data from other websites by default. We need third party libraries or standards to do this.

However, most JavaScript driven sites and applications need to get data from a database or data store in order to update the page and save data from user interactions.

The simplest way to do this is with a web standard called the Fetch API. The Fetch API lets us easily get data from another URL.

// Fetches JSON data from a website
fetch( `https://api.dev/data` )

The full code for the Fetch API involves a little more than this, but that is basically all we have to include in our JavaScript.

Since the Fetch API is not part of the JavaScript language, it relies on the browsers to offer support for it. So, we usually want to include a library like this fetch polyfill https://github.com/developit/unfetch that ensures everything will work in older browsers (and IE).

<!-- Link to Fetch API polyfill for support in all browsers -->
<script src="https://unpkg.com/unfetch/polyfill"></script>
<!-- Link to our own JS that can now use Fetch API -->
<script src="src/index.js"></script>

A Complete Example of the Fetch API Getting Data

Here is an example of the Fetch API in action. To practice we will get data from the Space X API, which provides data about recent launches of rockets.

// Fetch data from the Space X API
fetch( `https://api.spacexdata.com/v3/launches/latest` )
// Convert the data into JavaScript
.then(response => response.json())
// Now we can use the data
.then(data => {
// Log out the data
console.log(data)
// Add some data amd markup to the page
document.querySelector(`#app`).insertAdjacentHTML( `beforeend`,
`<h2>${data.mission_name}</h2>
<p>
${data.launch_year} -
${data.launch_site.site_name_long}
</p>`
)
})
// Log out any errors
.catch( error => console.error( error ));

Let's break down what is happening here:

  1. On line 2 we enter the URL the Space X API gives us in their documentation: https://github.com/r-spacex/SpaceX-API#usage.

  2. Then, line 4 takes the fetch request and converts it into JavaScript (Don't edit this line. It will go in most of the fetch examples).

  3. Line 6 is where we get the data in a format we can use.

  4. Then we can log out the data, line 9, to see what we have. This is always a good idea.

  5. On line 12 we get the div with an id of "app" and insert HTML inside of it

  6. Lines 13-17 create some markup to go on the page that includes launch data

  7. At the very end on line 22 we catch any errors that may be coming through

This pattern of fetching data from a URL, processing it into JavaScript and then doing something with the data using .fetch() and .then() is called a "Promise Architecture."

// Fetch Promise Architecture
fetch( `https://api.dev/data` )
// Convert the data to JavaScript
.then( response => response.json() )
.then( data => {
// Work with the data here
console.log( data )
})
.catch( error => console.error( error )

The primary problem with this promise architecture is that it does not follow the same flow as the rest of our code.

// This will log first
console.log( `First` )
fetch( `https://api.dev/data` )
.then( response => response.json() )
.then( data => {
// This will likely log third
console.log( `Second`, data )
})
.catch( error => console.error( error )
// This will likely log second
console.log( `Third` )

Anything that involves fetching an API will likely take some time. However, the JavaScript processor does not wait for for a fetch call to return. It simply continues to process the rest of the JavaScript on the page.

That is why in the example above the code at the bottom will likely log before the code inside the fetch call.

When this nonlinear behavior becomes a problem, we have Asynchronous Functions.

Waiting for API Fetches with Async Functions

If we want to have the JavaScript processor wait for a fetch call to return before proceeding with the rest of the code, we first wrap our code in an async function.

async function getData() {
// Make our fetch call here and wait for it's return
}

An async function is just like any other function, except that it will pause and wait anywhere it finds the await keyword before a fetch call.

async function getData() {
// Logs first
console.log( `First` )
// Waits for data
const data = await fetch(`https://api.dev/data`).then(res => res.json())
.catch( err => console.error( err )
// Logs second
console.log( `Second`, data )
// Logs third
console.log( `Third` )
}
// Fires off the entire process
getData()

You can see here that we have solved our flow problem and now the JavaScript processor will wait for our data before loading the rest of the app.

Sometimes we don't want our JavaScript to wait for certain content to load. For example, we can build out an entire page in JavaScript while we wait for the main content to display. A number of loading design patterns exist to keep the user engaged. In these cases, the first Fetch API Promise Architecture works fine.

However, sometimes we want to collect a bunch of data before doing something. Or we just don't want any action to take place before we know we have the data in our JavaScript. In these cases we will likely want to use the async functions with await before our fetch calls like in the example above.

It is also quite common to take our fetch call and place it into it's own function. The important thing to remember in this case is that the function must return a fetch call like so:

async function init() {
// Logs first
console.log( `First` )
// Waits for data
const data = await getData()
// Logs after data has returned
console.log( `Second`, data )
}
init()
function getData() {
// Return our fetch call
return fetch(`https://api.dev/data`)
.then(res => res.json())
.catch( err => console.error( err )
}

Note above that our new function must return a fetch call otherwise it will not work.

Examples of Fetching Different APIs

A number of APIs exist. JavaScripter, Todd Motto, has a great list of public APIs that you can look at: https://github.com/toddmotto/public-apis.

Regardless of what API you use, you will likely get back some JavaScript object with a bunch of nested properties that represent your data.

The first step when working with an API is log out your data and begin to drill through it, exploring what you have.

Over the next few examples, we will look at a few APIs so you can begin to see the similar process of fetching a URL for data, logging out that data, then trying to get data within the data.

NASA Photo of the Day

Here is an example of getting the NASA picture of the day. This API requires getting an API and adding it to the URL when you make a request. API Keys are quite common.

// This API requires you sign up and get an API key
const apiKey = `aXrmyfzBvWkpaQDP5EUnetn3Mn4eYfjakTi0523y`
const apiURL = `https://api.nasa.gov/planetary/apod?api_key=${apiKey}`
// Get the NASA Picture of the Data
async function getSpacePhoto() {
// Wait for the data
const data = await fetch( apiURL )
.then( response => response.json() )
.catch( error => console.error( error ))
// Log the data
console.log( data )
// Add the title and the picture to the page
document.querySelector(`#app`).insertAdjacentHTML( `beforeend`,
`<h2>${data.title}</h2>
<img src="${data.url}" alt="${data.title}" />`
)
}

When we log out the data on line 14 we find that we have a number of pieces of information. Two of those pieces of information are the title and url of the image. We can use this data when we create our HTML and add it to the page.

Giphy.com API

In this example below we are going to fetch GIFs from the giphy.com API, which also requires you to sign up for a free developer API and use it when making requests.

// Setup the URL with
const url_root = `https://api.giphy.com/v1/gifs/`;
const api_key = `fz6NfPnoHI4FZc5JvH7sR4clZAv2Y5HN`;
const search_term = `code`;
const url = `${api_root}search?q=${search_term}&api_key=${api_key}&limit=5`;
// Get the GIFs
async function getGIFs() {
// Wait for the data
const data = await fetch( url )
.then( response => response.json() )
.catch( error => console.error( error ))
// Log the data object inside the data object!
console.log( data.data )
// Create markup for each GIF
let markup = ``
data.data.map( gif => {
markup += `<img src="${gif.url}" alt="${gif.title}" />`
})
// Add the GIF markup to the page
document.querySelector(`#app`).insertAdjacentHTML( `beforeend`, markup )
}

One of the interesting thing we find with the Giphy API is that we have to type data.data in order to get our data! That is kind of weird, but shows us why it is always a good idea to log out our data!

Here is one more example of working with the Fetch API, just so you can continue to see what things are similar and what are different.

Github.com API

In this example we are going to use the Github API to pull together two API requests, one to get the user data and another to get a list of the users repos. It is common with some APIs that you might not be able to get everything you want with one API request.

Along the way we will add (and remove) Loading notices to let the user know a request is happening.

First, we create two functions. One to fetch our user data and one to fetch our repos.

const container = document.querySelector(`#app`)
const api_root = `https://api.github.com/users`
function getUser( username ) {
const url = `${api_root}/${username}`;
return fetch(url)
.then(res => res.json())
.catch(err => console.error(err));
}
function getRepos(username) {
const url = `${api_root}/${username}/repos?per_page=20`;
return fetch(url)
.then(res => res.json())
.catch(err => console.error(err));
}

Then we can create an async function that gets the user information, adds it to the page, and then goes off to get the repos and add them to the page.

This will ensure that the user is getting information back as soon as possible after the first request is done, rather than having to wait for both requests to complete.

async function renderUser(username) {
// Let the user know a fetch is happening
container.innerHTML = "Loading Profile...";
// Wait to get the user data
const user = await getUser(username);
console.log(user);
// Create markup with the user data
let markup = `
<div class="user">
<img src="${user.avatar_url}" alt="${user.name}" width="100" />
<h2><a href="${user.html_url}">${user.name}</a></h2>
<p>${user.bio}</p>
</div>
`;
// Replace the loading message with user markup
container.innerHTML = markup;
// Display a list for the repos with a loading message
container.insertAdjacentHTML( `beforeend`, `
<ul class="repos">
<li>Loading Repos...</li>
</ul>`
);
// Wait to get the repos
const repos = await getRepos(username);
console.log(repos);
// Create markup for the repos
markup = ``;
markup += repos
.map(repo => {
console.log(repo);
return `<li><a href="${repo.html_url}">${repo.name}</a></li>`;
})
// This line is needed if you ever notice commas between items
.join("");
// Replace the repo loading message with repo markup
document.querySelector(`.repos`).innerHTML = markup;
}
// Kick everyhing off for a user
renderUser(`zgordon`);

While you might not feel comfortable coding this from scratch yet, hopefully the code makes sense along with the comments.

We make one request, wait, and then load the data to the page. We then make another request, wait, and then load the data on the page.

API Keys

As we have seen with the examples above, some APIs require you to have an API key. This identifies requests coming from you and offers some level of security for the API server.

Some API keys are public API keys and can appear in your JavaScript code or repositories. Other keys are private API keys. These should not be stored in your JavaScript or public repositories where people can find them.

When working with private API keys, you will likely want to use some server side programming language like Node, PHP, Python, Ruby or something similar that will allow your private API key to be saved somewhere people cannot easily find it.

It is a bit beyond the scope of this book to demonstrate how to setup private API keys, however, if one is required for a specific API, their documentation should offer suggestions on how to set this up with your project.

A Note on Cross-Origin Resource Sharing (CORS)

When making API calls with JavaScript it is important to understand a term called cross-origin resource sharing or CORS.

When a JavaScript application makes an HTTP request to a server, that server has the ability to refuse to share the requested resource. When this happens you get what is called a CORS error that often looks something like this:

ERROR: Access to fetch at `https://api.requestedurl.com/`
from origin `https://yoururl.com` has been blocked
by CORS policy.

If you control the server you are making the request to, for example a WordPress site that you run, it is possible to reconfigure your server to allow for CORS. However, if you do not have control over the server you are requesting you will have to contact them to see if this is something you can have enabled.

Alternatives to the Fetch API

The Fetch API is the most recent resource available to us for making API calls with JavaScript, but it is not the only one. An older protocol is called XMLHttpRequest, which does something similar but with a different syntax and set of browser compatibility issues.

There are also many HTTP Clients, or special libraries built using the Fetch API or XMLHttpRequest, that also allow us to make API calls in different ways. Some APIs even have their own HTTP Clients that you have to use to interact with their APIs.

Axios is a very popular HTTP Client. It is built using XMLHttpRequest but looks very similar to the Fetch API. The primary benefit to using a client like Axios is it handles a lot of possible browser compatibility issues that exist for both the Fetch API and XMLHttpRequest.

Here is a simple example of Axios in action. Note that the Axios library must also be linked to before our own JavaScript.

// Make a request for a user with a given ID
axios.get('https://api.site.com/')
.then(function (response) {
// handle success
console.log(response);
})
.catch(function (error) {
// handle error
console.log(error);
})

There is a large chance as you continue to work with API calls and JavaScript that you will need or want to use an alternative to the Fetch API. While the syntax may change slightly between the Fetch API and other HTTP Clients, the basic principals of how to make API calls and use the data should remain the same.

Loading Animations

When we work with API calls in JavaScript we usually have a delay between when the page loads and the API call returns with the data. Or we have a delay between when a user takes an action, like clicks a button, and when the action is complete.

In these instances it is helpful to communicate to the user that something is happening so they are not left wondering. A common technique is to display a loading animation.

A simple loading animation can involve displaying an animated GIF or SVG file right before an API call and then removing it after the API call has completed.

Take a look at this simple flow below where we display a loading animation until the content is loaded.

async function init() {
// Load a spinner animation
loadSpinner();
// Get some data
const data = await fetch(`http://api.dev`).then(res=> res.json());
// Stop the loading animation
removeSpinner();
// Load the data
loadData(data);
}
init();
function loadData(data) {
document
.querySelector(`#container`)
.insertAdjacentHTML(`beforeend`, img);
}
function loadSpinner() {
// Get the path to an animated SVG
const spinner = `../assets/loader.svg`;
const img = `<img src="${spinner}" id="spinner" />`;
document
.querySelector(`#container`)
.insertAdjacentHTML(`beforeend`, `<div>${data}</div>`);
}
function removeSpinner() {
const spinner = document.querySelector(`img#spinner`);
spinner.parentElement.removeChild(spinner);
}

In the code above we load a spinner, the get our data, and once we have our data back we remove the spinner and load our data. This gives the user some awareness that something is happening and they should continue to wait.

There are more advanced approaches to loading animations, including libraries dedicated just to loading animations. However, this simple approach should help you get started.

Next Up

Now that we have learned about the Fetch API and seen it in action with a few different APIs, it is time to do some practice to ingrain what I have learned.

Through the practice exercises in the next chapter you will get comfortable hitting a number of different APIs and making Fetch API calls in different ways.

Then we will take everything we have learned up to this point and apply it to a large project.