Mark's Dev Blog

Random musings on React, Redux, and more, by Redux maintainer Mark "acemarke" Erikson
 Sponsor @markerikson

Practical Redux, Part 9: Upgrading Redux-ORM and Updating Dependencies

Posted on
#javascript #redux #react #create-react-app #npm #yarn #redux-orm

This is a post in the Practical Redux series.

Updates to app dependencies, use of Redux-ORM 0.9, and caching NPM dependency files

Intro 🔗︎

Greetings, and welcome back to the Practical Redux series! It's been a few months since the last post, but as promised, I still have a lot of topics I want to cover for this series.

Picking up where we left off: in Part 8, we added the ability to delete Pilot entries, used our generic entity reducer logic to implement "draft data" handling for the Pilot editing form, and added the ability to cancel and reset form edits. This time around, we have some housekeeping to do: we'll update Redux-ORM to 0.9 and discuss the migration and API changes, update our other dependencies to the latest versions, and look at ways to cache NPM dependencies for consistent project installation even if you're offline.

The code for this project is on Github at github.com/markerikson/project-minimek. The original WIP commits I made for this post can be seen in PR #10: Practical Redux Part 9 WIP, and the final "clean" commits can be seen in in PR #11: Practical Redux Part 9 Final.

I'll be linking to each "final" commit as I go through the post, as well as specific files in those commits. I won't paste every changed file in here or show every single changed line, to save space, but rather try to show the most relevant changes for each commit as appropriate.

Table of Contents 🔗︎

Using Redux-ORM 0.9 🔗︎

When I started the series, the current version of Redux-ORM was 0.8.1. Since then, there have been some noticeable changes. Author Tommi Kaikkonen built and released version 0.9, which includes a number of breaking changes to the API and the library's behavior. Sadly, he also no longer has time to maintain the library by himself, but a call for new maintainers resulted in a couple people being given full ownership of the library, and several others (including myself) were given commit rights. So, Redux-ORM is definitely alive and well.

Let's take a look at the major changes in Redux-ORM 0.9, then tackle upgrading Project Mini-Mek to use it.

Redux-ORM 0.9 Changes 🔗︎

One of the nice things about the Redux-ORM library is its documentation. Too many JS libraries have nothing more than a few example snippets in their README, but Redux-ORM has had a couple pretty good intro tutorials and decent API docs since the beginning.

The Redux-ORM 0.9 changes included a helpful migration guide for using 0.9. Since that document is pretty good as-is, I won't try to restate everything, but I will highlight the biggest changes:

  • The 0.8 Schema class has been renamed to ORM, due to conceptual changes in how it works internally. Some of the Schema methods have been replaced:
    • schema.from(state) is now orm.from(state)
    • schema.withMutations(state) is now orm.mutableSession(state)
    • schema.getDefaultState() is now orm.getEmptyState()
  • Session instances now apply changes immediately, instead of queueing up changes to be applied when you called session.reduce(). That means that after any change, like myModel.someField = 123, the session.state object will have been correctly immutably updated to reflect the current values (so session.state.MyModel.byId[id].someField would now be 123 right away). This simplifies complex reducer logic considerably, since you no longer have to worry about what updates are currently pending inside the session's queue and which have been applied. This also means that the session.state = session.reduce() trick I showed previously can be dropped, as the session basically does that for you automatically.
  • The prior APIs for creating an entities reducer and selectors from a schema have been moved into separate functions. I haven't yet used any of those myself, in either this series or elsewhere.
  • The QuerySet "flag" properties for someQuerySet.withModels and someQuerySet.withRefs have been removed, and you should now use toModelArray() or toRefArray() instead.
  • QuerySets are now lazy, so you can chain them together and only execute when needed
  • Model classes now can take declarations of expected field names. In 0.8, Model instances dynamically created ES5 properties for values that were in the underlying plain data object and for relational fields, each time an instance was built. In 0.9, you can add field declarations to your model class, and Redux-ORM can more efficiently handle those. (The field declarations are still optional if you prefer not to use them.)

In addition, 0.9.1 added a Model.upsert() method. As mentioned in Part 2, Redux-ORM didn't merge or de-duplicate items with the same ID the way Normalizr does. I had opened up an issue to discuss possible fixes, and the new upsert() method was added as a result. So, calling upsert() instead of create() takes care of things nicely.

Upgrading Project Mini-Mek to Redux-ORM 0.9 🔗︎

With those changes in mind, let's go ahead and migrate Project Mini-Mek to use the latest version of Redux-ORM. At the time of writing, that's 0.9.4. For this step, we'll update this version explicitly. I played around with both yarn add and yarn upgrade a bit. Both work, but yarn upgrade seemed to lock the entry in package.json to the given version even if I added the ^ in front of it. So, we'll do it with add:

yarn add redux-orm@0.9.4

That updates the contents of node_modules/redux-orm and modifies our package.json and yarn.lock files, so let's commit those:

Commit 2dd6024: Update Redux-ORM to 0.9.4

Now we can move on to actually modifying the application code to match. Fortunately, the changes aren't overly complicated, and mostly break down into a few edits repeated several times.

Commit 3c19945: Update Redux-ORM usage to work with 0.9.x

Let's go through examples of each change:

First, we need to replace creating a Schema with an ORM instead:

app/schema/schema.js

-import {Schema} from "redux-orm";
+import {ORM } from "redux-orm";

import Pilot from "features/pilots/Pilot";
import MechDesign from "features/mechs/MechDesign";
import Mech from "features/mechs/Mech";

-const schema = new Schema();
-schema.register(Pilot, MechDesign, Mech);
+const orm = new ORM();
+orm.register(Pilot, MechDesign, Mech);

Second, we need to use orm.getEmptyState(), our reducers that use a Session should stop using session.state = session.reduce(), and we should use toModelArray() instead of withModels:

app/reducers/entitiesReducer.js

import {DATA_LOADED} from "features/tools/toolConstants";

-import schema from "app/schema"
+import orm from "app/schema"

-const initialState = schema.getDefaultState();
+const initialState = orm.getEmptyState();

export function loadData(state, payload) {
    // Create a Redux-ORM session from our entities "tables"
-   const session = schema.from(state);
+   const session = orm.session(state);
    // Get a reference to the correct version of model classes for this Session
    const {Pilot, MechDesign, Mech} = session;

// skip ahead

    // Clear out any existing models from state so that we can avoid
    // conflicts from the new data coming in if data is reloaded
    [Pilot, Mech, MechDesign].forEach(modelType => {
-       modelType.all().withModels.forEach(model => model.delete());
-       session.state = session.reduce();
+       modelType.all().toModelArray().forEach(model => model.delete());
    });

-   // Queue up creation commands for each entry
+   // Immutably update the session state as we insert items
    pilots.forEach(pilot => Pilot.parse(pilot));

    designs.forEach(design => MechDesign.parse(design));
    mechs.forEach(mech => Mech.parse(mech));

-   // Apply the queued updates and return the updated "tables"
-   return session.reduce();
+   // Return the new "tables" object containing the updates
+   return session.state;
}

Third, our Model classes should be updated to add declarations for the fields they contain, and we can replace uses of create() with upsert():

features/pilots/Pilot.js

-import {Model, fk} from "redux-orm";
+import {Model, fk, attr} from "redux-orm";

export default class Pilot extends Model {
    static get fields() {
        return {
+           id : attr(),
+           name : attr(),
+           rank : attr(),
+           gunnery : attr(),
+           piloting : attr(),
+           age : attr(),
            mech : fk("Mech"),
        };
    }

    static parse(pilotData) {
        // We could do useful stuff in here with relations,
        // but since we have no relations yet, all we need
        // to do is pass the pilot data on to create() or upsert()

        // Note that in a static class method, `this` is the
        // class itself, not an instance
-       return this.create(pilotData)
+       return this.upsert(pilotData);
    }

Once we apply those edits consistently across the rest of the codebase, our application should now run correctly with no errors.

I haven't yet played with upsert() all that much, but at the moment I don't see any real downside to using it pretty much anywhere in place of create(). Note that because we're now using it, you could actually copy and paste the pilots.forEach(pilot => Pilot.parse(pilot)); line in loadData(), and those Pilot entries would load okay. You could also load a Pilot entry with an existing ID but a couple different values, and it should be merged in correctly (only overwriting the changed values).

Updating Dependencies 🔗︎

The React world has continued to move forward over the last few months as well. React is up to 15.6, Create-React-App hit the big 1.0 and is now at 1.0.10, and many other libs have been updated. Since we're updating things, tt would be good to get everything else up to date too.

Updating React-Scripts 🔗︎

As mentioned in Part 5, Create-React-App is really just the CLI tool that sets up your initial project. All the build system magic is taken care of inside the react-scripts package, and we only need to update that one dependency to take advantage of the latest improvements.

yarn add --dev react-scripts@latest

Commit c6621b3: Update react-scripts to 1.0.10

Updating Other Dependencies 🔗︎

We've got several other dependencies that need to be updated as well. We could list them all in a single yarn add or yarn upgrade command, but maybe we don't want to actually just automatically bump them all to the latest versions. Yarn includes a upgrade-interactive command that shows us the latest versions of each one, and lets us pick and choose which ones to update:

project-minimek> yarn upgrade-interactive
yarn upgrade-interactive
? Choose which packages to update. (Press <space> to select, <a> to toggle all, <i> to inverse selection)
 dependencies
 ( ) lodash                    4.16.6        ?  4.17.4
 ( ) react                     15.3.2        ?  15.6.1
 ( ) react-dom                 15.3.2        ?  15.6.1
 ( ) react-redux               5.0.0-beta.3  ?  5.0.5
 ( ) react-toggle-display      2.1.1         ?  2.2.0
 ( ) redbox-react              1.3.3         ?  1.4.3
 ( ) redux                     3.6.0         ?  3.7.1
 ( ) redux-devtools-extension  1.0.0         ?  2.13.2
 ( ) redux-thunk               2.1.0         ?  2.2.0
 ( ) reselect                  2.5.4         ?  3.0.1
 ( ) semantic-ui-css           2.2.4         ?  2.2.10
 ( ) semantic-ui-react         0.61.1        ?  0.71.0

Having said that, we might as well go ahead and bump all of them to the latest versions after all.

Commit e179c95: Update app dependencies to current versions

Removing the Custom Error Overlay 🔗︎

When I first set up the support for Hot Module Replacement in Part 3, I used a library called Redbox-React to show error messages. I'll temporarily throw an error inside a component's render method to show what it looks like:

That helps, but it could be better.

Since that time, Create-React-App has added a powerful and informative built-in error overlay. All we need to do to make use of it is remove our own error handling code from src/index.js. While we're at it, we'll also remove the dependency on redbox-react:

yarn remove redbox-react

Commit 9782e30: Remove custom error overlay in favor of CRA's built-in overlay

src/index.js

if(module.hot) {
-   // Support hot reloading of components
-   // and display an overlay for runtime errors
-   const renderApp = render;
-   const renderError = (error) => {
-       const RedBox = require("redbox-react").default;
-       ReactDOM.render(
-           <RedBox error={error} />,
-           rootEl,
-       );
-   };
-
-   // In development, we wrap the rendering function to catch errors,
-   // and if something breaks, log the error and render it to the screen
-   render = () => {
-       try {
-           renderApp();
-       }
-       catch(error) {
-           console.error(error);
-           renderError(error);
-       }
-   };
-
+   // Support hot reloading of components.
    // Whenever the App component file or one of its dependencies
    // is changed, re-import the updated component and re-render it
    module.hot.accept("./app/layout/App", () => {

Now if we throw the same error, the overlay looks like this:

Ahhh... much more readable, informative, and soothing. (Now all we need is a towel, and an electronic book with the words "Don't Panic" written on the front.)

Fixing PropTypes 🔗︎

If we restart our dev server and re-run the page, Create-React-App prints out a warning:

Compiled with warnings.

./src/common/components/FormEditWrapper.jsx
  Line 1:  React.PropTypes is deprecated since React 15.5.0, use the npm module prop-types instead  react/no-deprecated

Search for the keywords to learn more about each warning.
To ignore, add // eslint-disable-next-line to the line before.

As part of the preparation for React 16, the React team is working on splitting several capabilities out into separate packages. This includes React.createClass() and the PropTypes API. React 15.5 started showing warnings any time you try to use those out of the "react" package, Most of the major libraries in the React ecosystem have now been updated, but we need to do the same. Fortunately, this is an easy change:

Commit 8573ed7: Use prop-types lib instead of React.PropTypes

common/components/FormEditWrapper.jsx

-import React, {Component, PropTypes} from "react";
+import React, {Component} from "react";
+import PropTypes from "prop-types"

If we restart the server one more time, the warning should be gone.

Managing Dependency Packages for Offline Installation 🔗︎

Package managers are powerful tools. With just a config file and a command, we can download all the dependencies our application needs. Unfortunately, this also introduces many weaknesses and concerns. What happens if the latest version of a package breaks something for me? How can I know that I'm getting the same package contents every time? What if something happens to a package server, or I need to be able to install these packages offline?

The infamous left-pad incident and various Github outages have shown that these are very real questions. While the use of hashes and locked URLs can ensure that a package manager like NPM is actually seeing the same file each time, that doesn't help if the network is down.

There's been a few approaches suggested for handling NPM dependencies without needing a network connection. Using NPM or Yarn's local per-machine cache works, but only if you've downloaded the necessary packages on that machine before. A number of people have suggested that you check in your entire node_modules folder, but that's a very bad idea for a variety of reasons. That's potentially tens or hundreds of thousands of files taking up hundreds of megs on disk, AND that can include platform-specific binaries built post-install (like node-sass's inclusion of libsass). If you check in your node_modules on a Mac, there's a good chance that it won't work on Windows or Linux (and vice versa).

The most ideal solution is to actually check in the downloaded archives for each package. Since platform-specific artifacts like libsass are built after installation, you can safely clone a repo on any machine, install the packages from that per-repo cache, and have things built properly for that machine.

Yarn includes an "offline mirror" feature built in. As far as I know, NPM has not had this built in as a specific capability, but there's a third-party tool called Shrinkpack that can use an NPM "shrinkwrap" file file as the basis for caching package tarballs in the repo. Based on a recent Twitter conversation, it seems that this is also a future planned feature for a future NPM5 release, and that it's possible to sorta-kinda mimic that with a .npmrc file and the --prefer-offline flag for NPM right now.

So, let's set up a package cache for Yarn to work with. First, we need to create a .yarnrc file in the repo:

Commit 326f253: Add Yarn config file for an offline cache

.yarnrc

yarn-offline-mirror "./offline-mirror"
yarn-offline-mirror-pruning true

We'll set two values. yarn-offline-mirror is the folder where we want Yarn to save the package tarballs. By default, if you update package versions, Yarn will only add new files to that folder, but not remove outdated ones. If we set yarn-offline-mirror-prune to true, it will also remove old package tarballs to match what's currently installed.

Next, we need to actually force Yarn to reinstall everything. The simplest way is to rename the node_modules folder to something like node_modules.bak, then run yarn again. Now, if you look inside the offline-mirror older, you should see around 1000 archives, like react-15.6.1.tgz. We can git add the entire folder, and check them all in:

Commit XYZ: Commit dependency packages

And with that, anyone else who is using Yarn along with this project should be able to use exactly the same packages that I have right now, even if you try to install them while you're offline. In theory, anyway. For full disclosure, this is a feature that I've only partly played with myself, and I've actually spent the last couple days fighting Yarn trying to get a variation of this to cooperate in my project at work, without success. (Nitty-gritty details at yarn#3910.) So, seems really useful, but YMMV in practice.

Final Thoughts 🔗︎

This post was probably a lot less exciting than the last few parts, but in today's world, updating dependencies is a pretty important topic. I'm still learning a lot of these intricacies myself (and in some cases, actively struggling with the tools), so please don't take the steps I showed as absolute gospel.

The good news is that we can now get back to writing code, adding features, and demonstrating useful techniques. Next time, we'll look at how to implement Redux-driven modal dialogs and context menus.

Further Information 🔗︎

This is a post in the Practical Redux series. Other posts in this series: