📝 ✨ ~ Remote Astro Collections (sort of)

Mon, 11 Mar 2024 00:00:00 GMT

You may also watch us figure this approach out live on YouTube.

</aside>

After adding the "most popular" episodes section to our Astro podcast website the local DX took a nose dive!

Building the home page took 3789ms, that's almost 4 seconds 🤦‍♀️

Fetching all the analytics data from the Transistor API, looping through and calculating the total download numbers for each episode, and then sorting the episodes by popularity takes time.

We could probably optimize the code, but it would still slow down the build.
We could skip picking out the most popular episodes in development, but then it be hard to design that section.

And neither would solve the issue that fetching the data from the Transistor API also needs to happen in the single episode page template (/src/pages/episodes/[slug].astro) and the paginated archive pages (/src/pages/episodes/[...page].astro).

At this point, I really missed the Gatsby Data Layer 😢 I deep-dived into the Astro Collection docs hoping I'd missed the option to source remote data as an Astro Collection. I had not, but I did notice Astro collection can be of type data, meaning .json or .yml files in addition to type content meaning .md, .mdx and .markdoc files.

So I decided to explore fetching all the data, and calculating the total download numbers, ahead of time! Writing the response from the Transistor API as .json files to disk; one for each episode.

import { join } from "node:path";
import { writeFile } from "node:fs/promises";
import { fetchAllEpisodes, fetchAllEpisodeAnalytics } from "./transistor.js";

try {
  const DIR = join(process.cwd() + "/src/content/episodes/");

  const episodeData = await fetchAllEpisodes();
  const episodeAnalytics = await fetchAllEpisodeAnalytics();

  const episodeFilePromises = episodeData.map((episode) => {
    const analytics = episodeAnalytics.attributes.episodes.find((a) => {
      return parseInt(a.id) === parseInt(episode.id);
    });

    const totalDownloads = analytics.downloads.reduce(
      (acc, day) => acc + day.downloads,
      0
    );

    return writeFile(
      join(DIR + `/${episode.id}.json`),
      JSON.stringify(
        {
          ...episode.attributes,
          totalDownloads,
        },
        null,
        2
      )
    );
  });

  const files = await Promise.all(episodeFilePromises);
  console.log(`Episode entries written ${files.length} to ${DIR}.`);
} catch (error) {
  console.error(error);
}

And holy moly, it worked!
Build time for the home page went from 3789ms to 148ms 🎉🎉🎉

In addition to the build time improvement, I also got the added benefit of Astro's built-in collection validation. I decided to skip typescript for the data fetching code above, and rather rely on the collection schema to catch any errors.

import { defineCollection, z } from "astro:content";

const episodeCollection = defineCollection({
  type: "data",
  schema: z.object({
    title: z.string(),
    slug: z.string(),
    totalDownloads: z.number(),
    description: z.string(),
    embed_html: z.string(),
    // and more as we build out the site
  }),
});

export const collections = {
  episodes: episodeCollection,
};

My gut feeling is that we'll see support for remote collection in Astro pretty soon, but for now, I'm happy with the solution I've come up with.

Let me know if you have any questions or suggestions for improvements. I'm @raae on Twitter.

📝 ✨ ~ Two Week Build is on 🚀

Thu, 29 Feb 2024 00:00:00 GMT

I signed up for Amy's inaugural Two-Week Build Challenge. And there is still time for you to sign up!

Let me know if you need some added accountability. We could set up co-working sessions 🎉

How It Works

Pick Something to Build
Build it in 2 Weeks
Ship it

Rules and Guidelines

Have fun.
Don’t overthink it.
Don’t start until March 1, 2024.
You can build anything. Write an ebook. Design a UI library. Build a web application.
At the end of 2 weeks, ship what you have.

I'm building a, Astro website for our podcast Data in the Wild launching in April now that all episodes of season 1 are recorded.

And I plan on live streaming every day starting Monday, the 4th.

But if I fold or skip some, that is also okay...

👷‍♀️

📝 ✨ ~ Remix, Astro, or Eleventy?

Mon, 11 Sep 2023 00:00:00 GMT

We are, of course, hoping Gatsby gets a second wind, but as a content creator, it seems Gatsby will not get the piratical family and me anywhere these days 😬

So my question to you is:

Would you like to learn a new Jamstack framework with us, and if so, which one: Astro, Remix, or Eleventy?

Or is there something else entirely you would like to see from us?

All the best, Raae

PS: Remember to move your Gatsby Cloud sites within the next month(ish).

Posts tagged #astro

📝 ✨ ~ Remote Astro Collections (sort of)

📝 ✨ ~ Two Week Build is on 🚀

How It Works

Rules and Guidelines

📝 ✨ ~ Remix, Astro, or Eleventy?