Mark's Dev Blog

Random musings on React, Redux, and more, by Redux maintainer Mark "acemarke" Erikson
 Sponsor @markerikson

Blogged Answers: My Experience Modernizing Packages to ESM

Posted on
#javascript #esm #cjs #modernization #packages

This is a post in the Blogged Answers series.

Details on the painful experiences and hard-earned lessons I've learned migrating the Redux packages to ESM

Table of Contents 🔗︎

Introduction 🔗︎

For the last 8+ years, the JS ecosystem has been undergoing a slow transition towards using ES Modules ("ESM") as the default approach for publishing and using JS code. Similar to the Python 2->3 transition, this has been incredibly difficult and painful to deal with.

As a package maintainer, I want to make sure that my libraries are maximally compatible and usable in the widest array of environments I can feasibly support. Unfortunately, this also means that I've had to become familiar with the nuances and behavior quirks of a variety of different build tools and runtime environments

Early this year I started working on trying to update the package formatting for the Redux family of libraries to give them "full ESM compatibility". I think I've finally come up with a set of configurations that seem to work reasonably well, but it's been a struggle.

One of my biggest frustrations is that there is no single authoritative and comprehensive guide on "How to Publish a JS Package Correctly". I've repeatedly begged for some expert who actually knows what they're doing to write and publish such a guide. Ideally, it would cover things like what file formats to include, how to configure ESM/CJS interop and package.exports, dealing with TS types versions, ensuring tree shaking, checking for compatibility issues, how to properly support specific build tools and what they look for, and so on. I have found some guides (which I'll link below), but nothing that quite matches the breadth and contents I've been wishing for.

This post is not that "authoritative guide". It's a recap of what I've tried, and hard-earned lessons I've learned along the way. Based on the number of times folks have popped up and said "you're doing this wrong", I'm sure there's plenty of pieces I'm still missing :) But, I hope this information will be useful and informative even if it's not fully comprehensive and authoritative.

There's plenty of articles out there already recapping the history of the ESM spec, the arguments and decisions that have led to the current confusion and compatibility issues, and how we got into this mess. In the interest of keeping this a somewhat manageable size, I'll try to dig up links to a few of those and list them at the end, and focus primarily on my own experience and steps throughout this process.

Redux Packages Background 🔗︎

Packages and Configurations 🔗︎

At the end of 2022, I maintained and published these packages as part of the Redux org:

Each of these packages had their own development history, packaging setup, and build configuration. In general:

  • All the packages included ESM, CJS, and UMD build artifacts (with varying combinations of embedded process.env.NODE_ENV values, or pre-compiled to "development" or "production" versions)
  • All build artifacts used a .js extension
  • All the packages were being transpiled to ES5 syntax for IE11 compatibility
  • redux, react-redux, redux-thunk, and reselect were being transpiled with Babel and bundled with Rollup. RTK was built using a custom ESBuild wrapper script that did the bundling and primary transpilation, but also used tsc to lower ES2015 code to ES5.
  • All of the packages used the "main", "module", and "types" fields in package.json. None of the packages used the relatively new "exports" field for defining which build artifacts get loaded.
  • Most of the packages except RTK output build artifacts to different folders by type: dist for UMD, lib for CJS, es for ESM

Some examples from those package.json files:

{
  "name": "redux",
  "version": "4.2.1",
  "main": "lib/redux.js",
  "unpkg": "dist/redux.js",
  "module": "es/redux.js",
  "typings": "./index.d.ts",
  "files": ["dist", "lib", "es", "src", "index.d.ts"]
}

{
  "name": "react-redux",
  "version": "8.0.5",
  "main": "./lib/index.js",
  "types": "./es/index.d.ts",
  "unpkg": "dist/react-redux.js",
  "module": "es/index.js",
  "files": ["dist", "lib", "src", "es"]
}

{
  "name": "@reduxjs/toolkit",
  "version": "1.9.5",
  "main": "dist/index.js",
  "module": "dist/redux-toolkit.esm.js",
  "unpkg": "dist/redux-toolkit.umd.min.js",
  "types": "dist/index.d.ts",
  "files": [
    "dist/**/*.js",
    "dist/**/*.js.map",
    "dist/**/*.d.ts",
    "dist/**/package.json",
    "src/",
    "query"
  ]
}

RTK's setup was more complicated, because it has 3 separate entry points: @reduxjs/toolkit, @reduxjs/toolkit/query, and @reduxjs/toolkit/query/react. Note that RTK's package.json didn't list the two RTKQ entry points. Instead, there was an actual /query folder in the published package, with /query/package.json and /query/react/package.json files that in turn pointed over to the right artifacts in the dist folder. (This setup was the result of considerable experimentation, aka "it seems to work with Webpack and a couple other tools I think???").

While not directly relevant for the rest of the story, I'd like to give a shout out to two highly useful tools that I use as part of the publishing process:

  • release-it: automates the actual steps for publishing to NPM, including Git tagging and pushing
  • yalc: lets you do a full local "publish" of a package so that you can test out installing it into example projects. Avoids issues with symlinks (ie npm link), and tests out the real build and publish steps.

Issue History 🔗︎

In mid-2021, we received an issue reporting that RTK could not be loaded properly in both client and server code simultaneously. In early 2022, a similar issue reported that RTK could not be correctly imported in a .mjs ESM file, due to use of "module" but no "exports" field in package.json. Finally, another issue noted that RTK didn't work with TypeScript's new moduleResolution: "node16" option.

I'd previously asked around about the implications of adding "exports" to a package, and I'd been told that "this qualifies as a breaking change". That meant that I couldn't begin to consider doing it until the next major release for each of the packages. But, I had no idea when we'd get around to publishing majors. Redux 4.0 came out all the way back in 2018, and RTK 1.0 in late 2019. React-Redux 8.0 was more recent, in mid-2022.

The Redux core had actually been converted to TS in 2019, but we'd never shipped it. 4.x and its hand-written typedefs worked, and we had concerns about potential ecosystem churn from shipping a 5.0 major. We also had plenty of feature work to do with RTK.

We shipped RTK 1.9 in November 2022. After taking a couple month break, I finally sat down to start seriously working on trying to modernize our packages.

Early Attempts 🔗︎

I'd been stockpiling a list of tabs and articles around "publishing modern JS", ESM, and ESM/CJS interop for the last few years, knowing that this day would come eventually. (At last check, I had roughly 175 articles in that list!).

I reviewed several of those articles to figure out my initial steps. From what I read, I concluded that:

  • I needed to add "type": "module" to my package.json files, in order for Node and bundlers to detect the package as containing ESM files
  • I also needed to add the "exports" key to package.json, and add keys inside that listed the possible entry points and what build artifacts to use when imported in different environments

I'd seen mentions of using .mjs as a file extension to force Node to recognize a given file as being an ES Module. To be honest, I felt that looked ugly and ridiculous, and I did not want to use that extension at all.

My first attempt was RTK PR #3095: Migrate the RTK package to be full ESM. Per my PR description, it contained these changes:

This PR attempts to convert the RTK package from its existing "contains ESM and CJS modules, but not fully ESM", to be fully ESM with {type: "module"} and still support CJS:

  • BREAKING: Sets the main RTK package.json file to be {type: "module"}
  • BREAKING: Updates all entry point package.json files to use exports to point to the types, ESM file, and CJS file
    • I still have main and module in there because WHO KNOWS WHETHER THOSE STILL END UP GETTING USED BY SOME TOOLS OR NOT 🤷‍♂️
  • Updates the build script:
    • Fixed ESM compat on execution by replacing use of __dirname and fixing the Terser import
    • Switched all build targets to be "esnext", to ensure the output is untouched other than TS transpilation
    • Moved all CJS build artifacts to be nested a level deeper in each entry point in a ./cjs/ folder, ie, ./dist/query/cjs/
    • Added {type: "commonjs"} package files to those folders
    • Turned off the UMD build artifacts for now

The resulting package.json looked like this:

{
  "name": "@reduxjs/toolkit",
  "version": "2.0.0-alpha.1",
  "type": "module",
  "module": "dist/redux-toolkit.modern.js",
  "main": "dist/cjs/index.js",
  "types": "dist/index.d.ts",
  "exports": {
    "./package.json": "./package.json",
    ".": {
      "types": "./dist/index.d.ts",
      "import": "./dist/redux-toolkit.modern.js",
      "default": "./dist/cjs/index.js"
    },
    "./query": {
      "types": "./dist/query/index.d.ts",
      "import": "./dist/query/rtk-query.modern.js",
      "default": "./dist/query/cjs/index.js"
    },
    "./query/react": {
      "types": "./dist/query/react/index.d.ts",
      "import": "./dist/query/react/rtk-query-react.modern.js",
      "default": "./dist/query/react/cjs/index.js"
    }
  }
}

I did try testing out local builds in Vite, CRA4/5, Next, and Node, as well as running the publint tool. Things seemed to mostly work locally, so I put up the PR to see what would happen with CI

AND EVERYTHING EXPLODED!!!! 💣💣💣

I spent a few more hours fiddling with things, and reported my findings:

Well. The good news is I think the runtime code works.

Bad news is Jest is being a pain. In particular, something about the way it's importing redux-thunk makes the default import an object like {default}, which is not a middleware function, and thus the tests explode when we try to create a store.

I spent the last couple hours hacking around with the thunk exports and republishing it locally. Switching the thunk package over to not having a default export at all sorta helped, but now something about the "no dev middleware in prod" test is failing.

So, close, but can't even build this branch yet.

edit

where I left off yesterday was that:

  • redux-thunk has a default export, and Jest was now choking on that
  • I published redux-thunk@3.0.0-alpha that tried to convert it to ESM, but still had a default export included. That helped build with Next, but not our local Jest tests
  • I then did a local-only publish of redux-thunk that dropped the default export and only had named exports. That actually seemed to help, but one of our RTK tests that asserts NODE_ENV=production behavior for getDefaultMiddleware was breaking

Some folks in Reactiflux suggested that it might be worth investigating a switch to Vitest. I'd prefer not to do that if possible, on the grounds that migrating to a different test runner is not my priority or something I want to spend time on. On the other hand, it could also be something that would be beneficial in general and for this specific problem.

Migrating to Vitest 🔗︎

I really didn't want to burn time trying to migrate our entire test setup from Jest to Vitest. But, I'd heard plenty of positive comments about Vitest, including that it ran significantly faster and had better ESM support.

A couple days later I decided to give it a shot. To my pleasant surprise, the conversion was fairly straightforward.

I ended up with RTK PR #3102: Migrate RTK test suite from Jest to Vitest.

The main test setup made sense, and the jest.fn() -> vi.fn() swaps were straightforward. The biggest pain point I ran into was where we tried to mock the redux package to assert that configureStore was calling through to the core library. Had to do a bunch of fiddling with vi.mock() until something finally seemed to work. On the other hand, timer behavior seemed to work more consistently.

As part of this process, I found myself also needing to convert other auxiliary files in the repo to ESM syntax, such as Jest config files and build scripts with a .js extension.

I was able to get that PR passing, and merge it a couple days later.

Initial Alpha Testing 🔗︎

I published @reduxjs/toolkit@2.0.0-alpha.1 on January 21. This included the RTK packaging changes, as well as a similar change to redux-thunk, and modernized the build artifacts to no longer transpile any JS syntax and drop IE11 compat.

Of course, this did not work as well as I'd hoped :)

Mateusz Burzyński ( @andaristrake ) maintains several libraries, including Emotion and Preconstruct, and spends much of his time working on the TypeScript compiler for fun. He's an expert on many of the intricate nuances of packaging formatting.

When I announced alpha.2 on Twitter, Mateusz replied with several suggestions for tweaks (looking at both RTK 2.0 and Redux core 5.0 alphas):

no idea what dist/es/redux.mjs is for now if it's not even in the exports map

by using the types condition like this TS might always assume that this is a module, even if loaded/required from CJS target~. Since you don't have a default export... that's probably fine

I would include the module condition and point to the ./dist/es/index.js with it, this will allow the package to be only loaded once by bundlers, despite the consuming file's format (cjs vs esm)

get rid of process.env.NODE_ENV in your dist files, use development/production conditions to accomplish this stuff (probably best to make production the default and the development stuff an opt-in)

I saved those as an issue for reference.

Shortly thereafter, we received a couple of new issue reports back-to-back complaining of problems with the config in alpha.1/2:

When importing anything from @reduxjs/toolkit@2.0.0-alpha.2, typescript cannot resolve types when tsconfig.json's moduleResolution is set to "node16" or "nodenext". I found that adding the extension .js to the declaration imports resolved the issue.

The current config in the Alpha does not allow for consumption of the CJS bundle in modern version of Node/any tool that follows Node's module resolution spec, as it uses .js to refer to a CJS module despite the package setting "type": "module". If "type": "module" is set, .cjs is necessary in "exports".

Some bundlers do work around this and are more forgiving (whether or not that's a good thing is something else entirely), but this config will not work in Node and/or any environments that follow its resolution mechanism.

I was... not a happy camper:

I do honestly appreciate you filing the issue, but I am also legitimately getting angry at how messed up this whole situation is :(

I want to do right by my users and support the variety of build tools and environments I expect they'll be using for their apps.

But I can't do that if every single article and person is giving me contradictory instructions on what I'm supposed to do :(

Clearly this was going to take a lot of effort to figure out what was going on and catch possible errors.

Researching Better Configuration 🔗︎

Right at the same time, Devon Govett tweeted about improving Node+ESM support in a React-Aria package update.

I replied noting I was working on some similar efforts, and tagged Mateusz. Shortly after that I saw the issues get filed, linked and griped about them, and Mateusz again suggested removing type: "module".

I was feeling frustrated and begged him to publish a full blog post that would give details on his recommendations. Instead, he suggested we do a phone call and talk through things directly.

On February 27, Mateusz and I hopped onto a call along with Nathan Bierema (Redux DevTools maintainer). I saved the discussion notes in a gist:

Mateusz threw out a lot of thoughts around how ESM and CJS can get used, and questioned whether it even entirely makes sense to ship ESM at all. The information was useful in general, but it left me still feeling pretty confused about next steps.

Somewhere in that Twitter discussion, I got in touch with Andrew Branch (@atcb), a TypeScript team member who had implemented the new moduleResolution: "bundler" option for TS, and has been doing work on JS/TS ecosystem module behavior as preparation for writing. We set up a call on February 28.

Andrew gave me a rundown of how ESM works, how TS treats ESM and module import paths, and how Node and other tools determine if a file is actually ESM.

The TL;DR of that last part is roughly:

  • If you add type: "module", every file with a .js extension gets interpreted as ESM, period. .cjs files will be interpreted as CommonJS.
  • Alternately, if you don't have type: "module", .js files are treated as CommonJS. You can use .mjs to mark individual files as ESM.

There was also some discussion of whether or not we should be pre-bundling our TS typedefs or leaving them as individual someSourceFile.d.ts files in the published package.

Setting Up CI Checks for Packaging 🔗︎

Initial CI Setup 🔗︎

I'd suspected for a long time that I was going to end up needing to put together some kind of battery of example applications, each using different build tooling, to catch possible errors in packaging during PR CI checks.

After the alpha.2 issue reports, I reluctantly concluded I really needed to spend time setting up those CI checks before I did any more work on the actual package configurations.

As part of my initial testing, I had locally created a small example app that exercised all of RTK's entry points. It had a counter to exercise configureStore and createSlice from the core, a UI-agnostic RTKQ createApi endpoint, and a React-specific RTKQ createApi endpoint. I'd pasted that into several different project setups.

We already had our CI set up to pre-build a tarball containing the package contents from the current PR, and were using that to run our unit tests against the PR package version instead of our source code. I decided to try expanding on that to test these different apps against that PR build as well.

I copied the first couple example projects into a new $REPO/examples/publish-ci/ folder, and updated the GH Action workflow to matrix the folder names inside of /publish-ci/, install the PR build into each example, then build+test it:

I also wrote a small Playwright test that would check the page contents to verify it could change the counter, and that both API endpoints had fetched the right mock data, in order to ensure that the apps actually ran correctly.

I actually targeted the PR against 1.9.x on our master branch, to see how things worked with the existing package setup first.

I eventually ended up with example apps that covered:

  • CRA 4 (with Webpack 4)
  • CRA 5 (with Webpack 5)
  • Next.js (with Webpack 5)
  • Vite 4
  • Node in both CJS and ESM modes

There were plenty of other build tool combinations that probably ought to be checked, but this is a good start.

Are The Types Wrong? 🔗︎

Somewhere in this process, I had discovered that Andrew Branch had created a tool called Are The Types Wrong. It's a website that lets you pick a published NPM package version, or upload a .tgz file, and analyzes the package exports to report how TypeScript interprets the configuration and whether the JS files and TS typedefs match up correctly. It then shows all your detected entry points, and reports details on any mismatches and errors.

Here's an example of the report for RTK 2.0.0-alpha.2 ( https://arethetypeswrong.github.io ):

Are The Types Wrong - RTK 2.0-alpha.2 results

You can see that it detected all of RTK's entry points, and that most of the entry point + moduleResolution combinations look okay. But, moduleResolution: "node16" + some CJS environment apparently has multiple issues.

I really wanted to use this analysis in RTK's CI to help verify that any future PR changes would actually work correctly. I looked at the attw repo, and noted that Andrew had split it into core and website packages. But, the core logic wasn't yet published as a package.

I initially tried setting up a CI task that would clone the attw repo, and let me write a command-line script that would import the core logic and analyze the PR build artifact. That technically worked, but fortunately I was able to convince Andrew to publish the logic as an actual @arethetypeswrong/core package.

From there, I put together a CLI script that ran the core attw logic, collected the reports, and wrote it out as a console table to match the display on the website. I did this using the ink React CLI renderer (and probably spent a bit too much time fiddling with rendering tables in the console). The results ended up pretty good:

Are The Types Wrong CLI output

I then configured RTK's CI to call that as another check alongside building the example apps.

There was an existing thread asking for attw to add a CLI, so I offered mine up as a potential starting point. (Someone else later filed a PR to add a CLI. That CLI has now been published officially, and I need to get around to switching over to using that in CI instead of my homegrown script.

Packaging Updates, Round 2 🔗︎

I decided it was best to try updating the smaller packages that RTK depends on first.

I'd already had to publish a 3.0-alpha.0 for redux-thunk to alter its behavior. I decided I'd switch over and try making further updates with that.

Switching Build Tooling 🔗︎

redux-thunk is a single tiny source file about 20 lines long (plus some additional TS types). That made it a good starting point to mess with changing packaging.

I noted that it was still using Babel+Rollup for the build step. I decided I'd try using ESBuild instead. But, how should I use that?

We already had a custom ESBuild wrapper script over in RTK. I briefly considered copy-pasting that over to the redux-thunk repo, but decided it would be overkill.

I'd done some previous searching for other ESBuild wrappers. I did some more looking and decided to give https://github.com/egoist/tsup a shot.

tsup actually worked out pretty well! In a couple hours I had a simple tsup.config.ts file that generated the two artifacts I wanted. In this case the thunk code had no dev/prod conditional checks, so I kept it really simple - just a single ESM and CJS file apiece.

I also removed type: "module", and switched to using .mjs and .cjs for the artifacts to force ESM or CJS appropriately.

I updated the thunk package.json file to use those:

{
  "name": "redux-thunk",
  "version": "3.0.0-alpha.1",
  "main": "dist/cjs/index.cjs",
  "module": "dist/index.mjs",
  "types": "dist/index.d.ts",
  "exports": {
    "./package.json": "./package.json",
    ".": {
      "types": "./dist/index.d.ts",
      "import": "./dist/index.mjs",
      "default": "./dist/cjs/index.cjs"
    },
    "./extend-redux": {
      "types": "./extend-redux.d.ts"
    }
  }
}

Meanwhile, I ended up doing all the same Jest->Vitest conversion as the RTK repo.

UMD Build Artifact Changes 🔗︎

I also spent a bunch of time debating whether it was worth keeping UMD files or not. redux-thunk had shipped with a UMD bundle, primarily for use as a script tag (which I assumed was mostly being done in CodePens or similar examples).

I have repeatedly asked about whether to keep publishing UMD builds over the last couple years.

The closest I got to actual advice and answers were:

  • Fred K Schott: "HTML examples and code editors aren't even reasons to use UMD anymore, on their own. Ex: @CodePen ships with a built-in Skypack integration"
  • Marvin Hagemeister: "I think it's fine to skip UMD. All bundlers can consume ESM, and with sites like https://esm.sh that can be easily used in the browser."

So, I decided it was finally time to drop UMD builds from the Redux packages :)

Later, I decided the best replacement for UMD was to include another ESM-format build artifact that was pre-compiled to production mode and no longer had process.env.NODE_ENV references, so that it could be safely loaded in a browser. That way people could use it as a <script type="module"> and load it from the package as hosted by Unpkg or a similar CDN.

(I asked for feedback on use cases for keeping UMD as part of our alpha/beta release notes and have not yet received a single comment... but we all know that almost no one gives feedback on pre-release versions anyway 🤷‍♂️)

Webpack 4 Compat 🔗︎

In early March, I saw a tweet from Dominik Dorfmeister (maintainer of React Query) noting that Webpack 4 still has more downloads than Webpack 5. This is mostly due to existing projects that use Webpack internally, such as CRA 4, Storybook 6, Expo's web target, etc, as well as general ecosystem usage.

Unfortunately, Webpack 4 does not support the package "exports" field, and it also cannot properly parse code with optional chaining syntax. Finally, I also figured out that it doesn't like having a .mjs file in the "main" field either, and needs a .js extension instead.

Part of my goal for these major versions was to stop transpiling away any of our JS syntax, only transpile TS types, and only ship fully modern JS. However, I also care about giving our users a good out-of-the-box experience. It was clear that trying to only ship modernized code would cause problems for anyone still on Webpack 4.

I grudgingly decided that I would include an additional artifact specifically for Webpack 4 compat: ESM module format, transpiled to ES2017 syntax, and using a .js extension, and point to that in the "main" field.

Immer 10 Beta 🔗︎

Michel Weststrate, author of the Immer immutable update library, had said back in January that he planned to work on Immer 10 in the spring. The major planned updates were around performance and dropping legacy ES5 compat. But, Immer also had some similar packaging issues, including use of both default and named exports. Immer had shipped the addition of "exports" in a 9.x patch release, only for it to break many users. The change was tweaked and half-reverted immediately, leaving Immer with an odd package config.

I'd left a comment in January noting how I'd run into issues with packages having both default and named exports, so Michel opted to drop the default export in 10.0.

Once Immer 10 came out in beta, I updated the RTK 2.0 branch to depend on that.

TypeScript Declarations 🔗︎

Redux 4.x ships with a hand-written TS typedefs file. RTK 1.x has individual TS typedefs files per-source-file, generated from sources.

tsup has a dts: true option that will call tsc to generate typedefs, and bundle them together into a single file. That worked fine for Redux, but I ran into some kind of issue doing that for RTK. I eventually settled for sticking with individual per-source typedefs files in RTK.

At this point I had package configurations that passed the are-the-types-wrong checks, with one exception: a "FalseCJS" warning for the ESM artifacts in moduleResolution: "node16" mode.

I had some back-and-forth with Andrew Branch about this. The problem is that technically you should have separate TS typedefs for "my artifacts in CJS mode", and "my artifacts in ESM mode", because there can be actual differences in what's exported and how that gets accessed.

The approach Andrew recommends to fix this is to actually compile your project with tsc twice, with two different TS module settings, and ship two different sets of typedefs with .d.mts and .d.ts extensions to match your .mjs and .cjs/js artifacts.

Unfortunately, no build tool that I knew of at that time did this by default, and the idea of shipping 99%-duplicate typedefs bothered me. So, I opted to not try to fix this "FalseCJS" issue for our packages (at least for the time being).

Andrew Branch later filed a PR for tsup that tries to actually output one typedefs file per output format. As of writing this post, I have not yet actually tried this out myself, but I'll give it a shot later and see what happens.

Note that Andrew currently has a very long gist with his WIP comprehensive documentation on how TS interprets module formats, as well as articles in the are-the-types-wrong repo documenting all of the issues it can find.

Round 2 Results 🔗︎

I published both redux@5.0.0-alpha.4 and @reduxjs/toolkit@2.0.0-alpha.4 in early April, with a set of package configurations that I was fairly sure should actually be "correct":

I ended up with these configurations:

  • Redux:
{
  "main": "dist/cjs/redux.cjs",
  "module": "dist/redux.mjs",
  "types": "dist/redux.d.ts",
  "exports": {
    "./package.json": "./package.json",
    ".": {
      "types": "./dist/redux.d.ts",
      "import": "./dist/redux.mjs",
      "default": "./dist/cjs/redux.cjs"
    }
  }
}
  • Redux Toolkit:
{
  "module": "dist/redux-toolkit.legacy-esm.js",
  "main": "dist/cjs/index.js",
  "types": "dist/index.d.ts",
  "exports": {
    "./package.json": "./package.json",
    ".": {
      "types": "./dist/index.d.ts",
      "import": "./dist/redux-toolkit.modern.mjs",
      "default": "./dist/cjs/index.js"
    },
    "./query": {
      "types": "./dist/query/index.d.ts",
      "import": "./dist/query/rtk-query.modern.mjs",
      "default": "./dist/query/cjs/index.js"
    },
    "./query/react": {
      "types": "./dist/query/react/index.d.ts",
      "import": "./dist/query/react/rtk-query-react.modern.mjs",
      "default": "./dist/query/react/cjs/index.js"
    }
  }
}

I also included the browser-focused ESM artifacts as well, although they aren't explicitly listed in there. (I may try to bring them back under the "browser" or "unpkg" keys - will need to do more research there.)

I also kept the nested entry point package.json files in the RTK package as part of the Webpack 4 compat.

These configurations seem to build and run in all of the sample projects I had configured, although that didn't include a React Native project.

Other Package Updates 🔗︎

I later applied the same package updates to reselect@5.0.0-alpha.0.

I wasn't originally planning to do a major version release for React-Redux. However, after seeing some other potential breaking changes, I concluded it was worth doing a React-Redux v9.0 major to include the packaging updates, handle TS types changes from Redux 5, and stop using the React 18 useSyncExternalStore shim by default.

As of this writing, I have a PR open to update React-Redux's packaging, but had run into some TS issues and didn't get back to trying to fix those.

Updating Immer's Packaging 🔗︎

I did some perf checks on Immer 10 beta, and saw that it was significantly faster. However, I also noted that somehow Immer 10 beta seemed bigger than Immer 9. I pulled down the Immer 10 beta PR branch and ran some build size comparisons.

After discussion, a part of the increase was due to Immer's Map/Set support now being enabled by default, and Michel agreed to revert that since Redux doesn't need it and RTK is one of the biggest users of Immer. However, even with that change, Immer 10 beta was still bigger than Immer 9.

I dug into Immer's build system, and found out that it was still using the mostly-dead tsdx build tooling package and targeting ES6 syntax. This was adding a bunch of polyfills and dead code. I also found some issues with the sourcemaps and the listed bundles.

Since I'd just spent a bunch of time updating the Redux and RTK build configs, I volunteered to port those changes over to Immer as well. I quickly put together a proof-of-concept, and saw noticeable reductions in Immer's bundle size. I put up a PR for those Immer changes, and Michel merged that as part of v10.

Problems with Next.js and React Server Components 🔗︎

In early May, Next 13.4 was released. The headline feature was that the new "App Router", which is based on React Server components, was now considered stable and ready for production. As part of that, Next's CLI defaulted to creating new projects with the App Router and /app folder enabled out of the box, and the docs were updated to teach use of the App Router and React Server Components as the default.

We soon began getting a stream of new issues filed against React-Redux and Redux Toolkit, complaining that React-Redux didn't work correctly. The main reports were errors being thrown because React.createContext() was null or useLayoutEffect not existing in an RSC environment.

Meanwhile, my Redux co-maintainer Lenz Weber-Tronic, who also works on Apollo Client for his day job, had spent the previous couple months doing research into how to use client-side state management and data fetching libraries with RSCs.

One issue he'd run into was how to use RTK Query's createApi, which can generate React hooks, on both server and client, and he'd filed a React issue asking for suggestions on how to handle this.

He also wrote a long RFC for how to integrate Apollo and Next 13, which later led to publishing an experimental Apollo + Next interop package.

In mid-June, an Apollo user filed an issue reporting that Apollo broke with Next 13.4.6-canary.2. That was fixed shortly thereafter in another canary build, but the issue spawned a long and frustrated discussion.

In that thread, a Next dev reported that Next was now finding the ESM artifact, could thus better analyze what was being imported, and the use of client code in server components was now being considered an error. They mentioned: "If apollo client is going have server components solution then it needs to have a separate "react-server" export condition that only contain the server only exports".

After reading that, I chimed in and noted that RTK Query has a mixture of both UI-agnostic and React-hooks-based code, and users might want to use createApi on both server and client. I also pointed out that we pre-bundle RTK's artifacts, and I don't know how to further split those out just to satisfy these Next/RSC-imposed constraints.

Seb Markbage, the React team's long-time lead architect (and now working on Next at Vercel), replied: "The user of your api can still use the same api, as long as you publish an optimized version of the inner implementation that excludes the unnecessary code branches."

That led to a strongly worded debate. Lenz pointed out that this was asking the entire ecosystem to make potentially breaking packaging changes, especially since many packages like Apollo still don't have an "exports" declaration. That seemed to be a surprise to Seb, who suggested an incredibly hacky workaround of import * as React to fool their static analyzer.

I got extremely frustrated reading this exchange, and replied: "I've spent months trying to upgrade our packaging, and now you're telling me I have to do more work just to keep our code from breaking in RSC environments. This is very demoralizing.".

The conversation in that particular thread died down, but also coincided with a whole bunch of debate threads about RSCs on Twitter and Reddit.

A few weeks later, Lenz published an extensive post titled My take on the current React & Server Components controversy. In it, he noted that we think RSCs are a very useful technology, but:

  • It's much harder for us to help our users, and they're filing a lot of new support issues
  • There's a lot more about React (and Next) we now have to understand
  • It's now much harder to maintain and publish a library that works with React
  • It feels like there's been very poor communication from the React team about the status of RSCs and the use hook, and little discussion on how this will impact the ecosystem

He also listed a number of possible APIs that would help libraries better deal with RSC environments and data fetching.

Lenz's post received lots of strong positive feedback from folks agreeing with the list of issues and suggestions, and expressing sympathy.

We never really got any actual response from the React team around any of those API suggestions, or how to properly publish packages that cooperate with RSCs. I did get some outreach from some folks in the React org who are trying to work on adding official docs around RSCs, and was able to pass on a lot of community feedback around RSCs, marketing, and usage concerns.

The Next docs did get a page covering how to wrap third-party context providers with "use client", although that feels like a bit of a band-aid.

Venting 🔗︎

After spending so much time on these changes, I was getting fairly frustrated at the sheer number of things I was having to keep track of.

In late April, I tweeted:

Things I have to keep in mind when publishing a library in 2023:

  • Build artifact formats (ESM, CJS, UMD)
    • Matrixed with: dev/prod/NODE_ENV builds
  • Bundled or individual .js per source
  • exports setup
  • Webpack 4 limits
  • TS moduleResolution options
  • User environments
  • Behavior differences between bundlers
  • Node ESM/CJS modes
  • TS typedef output (bundled? individual? .d.ts, or .d.mts?)
  • Edge runtimes?
  • And now React's new "use client" and RSC constraints
  • All of this for upstream deps too

This is getting utterly ridiculous :(

I don't know how anyone is supposed to be able to keep up with all these possible configuration changes, edge cases, runtime environments, and conflicting constraints.

And there are no actual comprehensive guides on how to do this stuff. Everyone's cargo-culting from others.

I'm trying to do right by our users and publish packages that work in as many environments as reasonably possible, but this is incredibly frustrating to deal with.

It's a miracle anything about this ecosystem works at all.

This struck a nerve. The tweet went pretty viral (for me), with dozens of retweets, quotes, and replies. It also got linked in some newsletters as well.

Similarly, after I wrote my comment about "feeling demoralized" in response to RSC changes, I followed that up with another tweet linking the discussion and venting:

As a library maintainer in the React ecosystem, I'm getting pretty frustrated by the churn around React Server Components.

Really starting to question whether the touted benefits are worth the pain being inflicted on lib maintainers

That tweet also spread pretty widely and got a lot of reactions as well.

A few days later, I had a follow-up thought:

I find it ironic that both of my recent tweets a/b frustrations dealing with JS ecosystem churn as a lib maintainer (package setup, RSCs) have gone viral.

I normally try to stay positive and not gripe much publicly.

I guess other folks feel or sympathize with those frustrations

As one reply noted: "I think people know how patient and positive you are towards what you're doing and the ecosystem. If something makes you frustrated that means that something BIG isn't working right or it's wrong".

Final Thoughts 🔗︎

I usually like to write blog posts like this after some release is done, or some discussion has come to an end, when it feels like that story is complete and there's some concrete takeaways and answers to write about. It'd be great if I could say "hey, RTK 2.0 is published, and it includes a set of packaging updates that I know are correct and work everywhere".

Sadly, that's not the case here.

This has been an extremely busy summer for me across work, Redux, and personal time, and I've realized I'm dealing with some burnout. But, also, I started this post at the beginning of June coming back from React Summit, and have just now gotten back to it a couple months later to get it wrapped up.

Where Do Things Stand Today? 🔗︎

Here's the state of things as of today:

Lessons and Takeaways 🔗︎

I've said many times that I still feel like I barely know anything about the topics of module packaging and publishing types. In practice, I'd guess that I do actually know more than most JS devs, because I've had to spend so much time trying to work on this (both over the years, and this year specifically). But, given the insane complexity, myriad of conflicting requirements, and rapidly changing list of things to keep track of, I think it's pretty understandable that I still feel like a clueless imposter about all of these topics.

So, what have I actually learned?

  • I have a set of package configurations that seem to work in most bundlers and build tools right now, and seem to have valid ESM/CJS packaging
  • Publishing TS typedefs properly adds another level of complexity on top of that
  • Pre-bundling your JS build artifacts sidesteps potential issues around having to have ".js" in file imports, and seems to work okay with tree-shaking...
  • But then can lead to other problems when something like this "split out your client code for RSC scenarios" thing comes up
  • It's almost impossible to keep up with all the different tools and constraints that are out there, and they keep changing
  • Similarly, there just aren't enough official, comprehensive resources on publishing packages. I found some decent guides ( such as The Modern Guide to Packaging Your JS Library , The React Library Guide (draft) , and Publishing Modern JS ), but it still feels like so much of this knowledge is fragmented, scattered, and conflicting. There's especially a need for guidance on what package formats+artifacts you ought to be shipping to cover different kinds of usage scenarios.
  • The ecosystem desperately needs better tooling to help automate this process.
    • tsup seems to be pretty good, and I've seen some other tools that claim to help with shipping dual ESM/CJS packages that I haven't tried, but it just feels like so much of this could be automated.
    • Similarly, it feels like there's a need for some kind of "test-your-lib-against-a-range-of-build-tools As A Service" tool. I set up a bunch of sample Redux projects with a variety of build tools, and I saw react-aria did something similar, but this feels like the kind of setup that could be commoditized somehow and help lib authors verify that their packages work correctly across the ecosystem.
  • React Server Components are a useful concept and tool, but it sure feels like this is a major disruption that's going to break a lot of the ecosystem. I understand the React team's comments that "nothing about client React changes, this is all additive"... but at the same time it's a massive increase in mental overhead and use case complexity that both users and lib authors have to deal with.
    • There's been effectively no outreach from the React team to the library ecosystem about how to deal with packages and RSCs. To be fair, we did have a call with Dan+Andrew, Dan reviewed Lenz's RFC, and there's been some discussion in issues. But there was no announcement or warning about Next throwing errors on client imports, no table of "what's in each canary version?" available, and no kind of "guidance to lib authors for RSC compat" post or doc published.
  • The overall ecosystem CJS/ESM transition has been a long and ongoing nightmare, and it shows no signs of ending any time soon. Just recently I saw a pair of dueling posts that argued "CJS is hurting JavaScript", and "CommonJS is not going away". Clearly this is a problem we're going to be stuck with for years.

Future Steps 🔗︎

We still need to finish up Redux Toolkit 2.0, but that's going to take a while. Just trying to make sure all the packages are fully up to date with the packaging changes, cross-reference each other's versions, and that all the TS types work, is time-consuming. We probably have more code-related changes to make too, but we don't have a solid list of what else must be in this set of majors.

Additionally, all of us maintainers are dealing with trying to balance jobs, motivation, time, and real life, and that means Redux work is a lower priority for us right now.

It's been an exhausting year so far dealing with all these issues. We still intend to get out the set of major releases (eventually - absolutely no ETA promises here), and I'm hopeful that these new versions will be a benefit to the ecosystem and significantly improve compatibility.

I sure hope that all this time and effort will have been worth it.

Further Information 🔗︎

This is a post in the Blogged Answers series. Other posts in this series: