afterChange collection hook and drafts

[cbratschi]

Link to reproduction

private repo

Describe the Bug

The previousDoc value always returns either the last published version of the document or the last saved draft. This makes it difficult to react on field changes between published versions. For instance we want to add redirects whenever a slug changes but comparing previousDoc.slug to doc.slug will be wrong whenever a draft version was created in between.

The problem is there is no workaround available because in afterChange the last published version is already overwritten if the user publishes the changes.

A solution would be to add a originalDoc property which is equal to the last published version or last available draft if the document was not published before. This would make state handling much easier.

The documentation should be extended to include all necessary details.

To Reproduce

1. create collection using draft
2. add afterChange hook
3. monitor previousDoc value

Payload Version

2.3.1

Adapters and Plugins

db-mongodb, bundler-webpack, live-preview

[cbratschi]

Have the same problem in a revalidate hook for Next.js. In afterChange it's impossible to distinguish the following state changes:

• published (last saved) to draft
• draft (last saved) to published

Therefore every afterChange call leads to a revalidation i.e. much too often while editing because of the intermediate draft calls.

[cbratschi]

A partial workaround is to switch to the beforeChange hook and read the last saved document through the local API:

//Note: only works here, afterChange hook runs too late!
if (originalDoc._status === 'draft') {
    //get last published
    originalDoc = await payload.findByID({
        //@ts-ignore
        collection: collection.slug,
        id: originalDoc.id, //Note: data contains no id
        depth: 0,
        draft: false,
        locale: lang,
        //Note: needed or falls back to default language!
        //Note: returns undefined value on first save
        fallbackLocale: lang
    });

    //Note: right now always getting a document but could be a draft
    if (!originalDoc || originalDoc._status === 'draft') {
        if (DEBUG) {
            console.log('-> could not find last published document');
        }

        return;
    }

    if (DEBUG) {
        console.log('-> read last published document');

        //debug
        //console.dir(originalDoc);
    }
}

However, this has some drawbacks and won't work for revalidation hooks. Here the document has to be saved first.

findById() has some strange behaviors:

• Draft query is set to false but still getting a draft document if no document was published yet.
• We use a default locale and a fallback in the Payload settings. To know a translation exists, we have to set fallbackLocale to the locale value. In this case we get undefined values being returned or values being pre-processed by other hooks. I would prefer having an API which does not return invalid documents in case a translation does not exist yet.

Update: tried "fallbackLocale: 'none'" but this made no difference.

[JarrodMFlesch]

@cbratschi I am trying to follow along with your issue here. I believe you could accomplish this by writing to the req's context in a beforeChange hook and reading from it in the afterChange hook:

// beforeChange hook

let transitionState: 'fromPublishToDraft' | 'fromDraftToPublish' | null = null
if (originalDoc._status === 'draft' && data?._status === 'published') {
  transitionState = 'fromDraftToPublish'
} else if (originalDoc._status === 'published' && data?._status === 'draft') {
  transitionState = 'fromPublishToDraft'
}

req.context = {
  ...req.context,
  transitionState: transitionState || undefined, 
}

And then in your afterChange hook you could read from the req.context and determine if the document is transitioning from publish to draft or draft to publish or neither.

Does that sound like it would assist you with what you are looking to accomplish?

[cbratschi]

@JarrodMFlesch the situation is more complicated than that but the context could be used to pass data available at beforeChange to afterChange.

The follow transitions happen:

• edit a (original) document
• modify content
• document is saved as draft
  • beforeChange: originalDoc is the original document (correct naming)
• save document
  • beforeChange: originalDoc point to the last draft (points to the last saved document, not the original one)

We use the code listed above to read the real original document in beforeChange (see #4405 (comment)).

A workaround would be:

• beforeChange
  • read original document from database
  • pass original document to context
• afterChange
  • get original document from context (use for business logic)

[DanRibbens]

This feels like a tricky edge case, or possibly a new feature that makes this easier. If we changed orginalDoc to behave the way you're expected it could have unexpected complications to other project's use of hooks.

You should use payload.findVersions and add your filters to return only published documents to do your comparison against.
Alterntively, you could use the db adapter versions models directly. Something like this should give you what you what: payload.db.versions[slug].find({parent: {$eq: ${originalDoc.id}}, "version._status": { $eq: "published"}, { sort: { updatedAt: -1 }, limit: 1}.

If you're using postgres this will be different obviously, but it can be handled similarly.

I'm open to other ideas about how to improve the API. It seems like maybe we could look at adding an afterPublish hook or add previousPublished docs to the hooks for versioned collections.

Would love to hear what others think.

[cbratschi]

Had the same problem again while updating the revalidation handling for Payload 3:

Create new draft:

• getting _state 'draft'
  • getting error from Next.js: revalidate collection pages 67596e49168ed4c653ed838f failed: Route /admin/[[...segments]] used "revalidateTag pages.67596e49168ed4c653ed838f.de" during render which is unsupported. To ensure revalidation is performed consistently it must always happen outside of renders and cached functions. See more info here: https://nextjs.org/docs/app/building-your-application/rendering/static-and-dynamic#dynamic-rendering

Edit the draft:

• _state stays 'draft'

Publish:

• _state now 'published'

Edit:

• First autosave:
  • _state changes to 'draft'
  • previousDoc _state is 'published'
• Second autosave:
  • doc and previousDoc _state is 'draft'

Basically revalidate should only occur if the document gets published, a published version gets updated or the document gets unpublished. Draft changes should never trigger revalidates.

Looking now for a better solution. We have a couple of hooks which need to monitor this transition.

[cbratschi]

Well got a working version of a custom hook after trying several different implementations. I have to use req.query to detect draft and autosave updates. But this is probably not safe if the document operations are invoked through the local API and not network requests. Here it would be better to pass the draft arguments to more hooks:

https://github.com/payloadcms/payload/blob/main/packages/payload/src/collections/operations/updateByID.ts#L123

One left big issue is that transaction are always committed after all hooks are called. A payload.find() call in any of the hooks will therefore always return outdated data. For instance we are calling revalidateTag() which can anytime rebuild pages and if by chance (race condition) it reads outdated data, the fresh content will never be visible. This should be changed. There is a ticket somewehere.

Clicking unpublish does not change the state of the button. Will open a new bug report later.

Here is my code:

import type { CollectionConfig, CollectionSlug, Config, DataFromCollectionSlug } from 'payload';

//flags
const DEBUG = true; //cbxx

/**
 * Publish action.
 */
type PublishAction = 'published' | 'unpublished' | 'updated' | 'deleted';

/**
 * After publish hook.
 */
type AfterPublishHook = (args: AfterPublishArgs) => void | ((args: AfterPublishArgs) => Promise<void>)

/**
 * After publish hook arguments.
 */
interface AfterPublishArgs {
    /**
     * Publish action.
     */
    action: PublishAction

    /**
     * Publish data.
     */
    doc: any

    /**
     * Previous published data.
     */
    previousData?: any
}

/**
 * Redirects plugin configuration.
 */
export interface PluginConfig {
    //no params so far
}

/**
 * Publish hook plugin.
 *
 * Reason for this plugin:
 *
 *   - Payload tracks changes between the latest draft but not between published version.
 *     Therefore its difficult to track state changes, such as publish, unpublish, delete
 *     and updates of published versions.
 *
 * Adds a custom hook:
 *
 *   - custom.hooks.afterPublish
 *
 * @param _pluginConfig
 * @returns
 */
export function publishHookPlugin(_pluginConfig: PluginConfig) {
    //modify configuration
    return (config: Config) => {
        //modify collections
        const collections = (config.collections || []).map(collection => {
            //check custom hooks used
            const hooks = collection.custom?.hooks;
            const afterPublish = hooks?.afterPublish as AfterPublishHook[];

            if (!afterPublish?.length) {
                return collection;
            }

            if (DEBUG) {
                console.log(`Using publish hooks: ${collection.slug}`);
            }

            //create new collection data
            return <CollectionConfig>{
                ...collection,

                hooks: {
                    ...(collection.hooks || {}),

                    //before save
                    beforeChange: [
                        ...(collection.hooks?.beforeChange || []),

                        /**
                         * Handle before change.
                         *
                         * @param param0
                         * @returns
                         */
                        async ({ data, originalDoc, operation, req, context }) => {
                            if (DEBUG) {
                                console.log(`-> beforeChange: ${collection.slug} ${operation} ${req.locale}`);

                                //debug
                                //console.dir(req);
                                //console.dir(context);
                                //console.dir(req.query);
                            }

                            //get currently published document
                            let lastPublished: DataFromCollectionSlug<CollectionSlug> | null;

                            if (operation === 'create') {
                                //first document version
                                lastPublished = originalDoc;
                            } else if (originalDoc._status === 'published') {
                                //last version was published
                                lastPublished = originalDoc;
                            } else {
                                //get last published
                                if (!req.locale) {
                                    console.error(`language missing: ${collection.slug}`);

                                    return data;
                                }

                                lastPublished = await req.payload.findByID({
                                    //@ts-expect-error correct collection slug
                                    collection: collection.slug,
                                    id: originalDoc.id, //Note: data contains no id
                                    depth: 0,
                                    draft: false,
                                    locale: req.locale,
                                    //Note: returns undefined value on first save (with draft or published status)
                                    fallbackLocale: false,
                                    disableErrors: true
                                });

                                if (DEBUG && lastPublished) {
                                    console.log('-> read last published document');

                                    //debug
                                    //console.dir(lastPublished);
                                }
                            }

                            //save in context
                            context.lastPublished = lastPublished;

                            return data;
                        }
                    ],

                    afterChange: [
                        ...(collection.hooks?.afterChange || []),

                        /**
                         * Have to check which version is now published to trigger hooks.
                         *
                         * @param param0
                         * @returns
                         */
                        async ({ doc, operation, req, context }) => {
                            if (DEBUG) {
                                console.log(`-> afterChange: ${collection.slug} ${operation} ${req.locale}`);
                            }

                            /*
                             * Ignore draft updates.
                             *
                             * Autosave:
                             *
                             *   - { draft: 'true', autosave: 'true', locale: 'de' }
                             *
                             * Publish/unpublish:
                             *
                             *   - { depth: '0', 'fallback-locale': 'null', locale: 'de' }
                             */
                            const query = req.query;

                            if (query.draft === 'true' || query.autosave === 'true') {
                                if (DEBUG) {
                                    console.log('-> ignore autosave or draft');
                                }

                                return doc;
                            }

                            //check state transitions
                            const lastPublished = context.lastPublished as Record<string, any> | undefined;
                            let action: PublishAction | undefined;

                            if (doc._status === 'published') {
                                //publish or updating published document
                                action = lastPublished?._status === 'published' ? 'updated':'published';
                            } else if (lastPublished?._status === 'published') {
                                action = 'unpublished';
                            }

                            //ignore draft updates
                            if (!action) {
                                return doc;
                            }

                            if (DEBUG) {
                                console.log(`-> action: ${action}`);
                            }

                            //call hooks
                            //Note: database operation happens befor (see https://github.com/payloadcms/payload/blob/main/packages/payload/src/collections/operations/updateByID.ts#L357) but commit later (see https://github.com/payloadcms/payload/blob/main/packages/payload/src/collections/operations/updateByID.ts#L477)
                            const args: AfterPublishArgs = {
                                action,
                                doc,
                                previousData: lastPublished
                            };

                            for (const hook of afterPublish) {
                                const res = hook(args);

                                if (res instanceof Promise) {
                                    await res;
                                }
                            }

                            return doc;
                        }
                    ],

                    /*
                    beforeDelete: [
                        //Note: return value ignored
                        async ({ id }) => {
                            if (DEBUG) {
                                console.log(`-> beforeDelete: ${collection.slug} ${id}`);
                            }
                        }
                    ],
                    */

                    afterDelete: [
                        /**
                         * Send delete action to hooks.
                         *
                         * @param param0
                         * @returns
                         */
                        async ({ id, doc }) => {
                            if (DEBUG) {
                                console.log(`-> afterDelete: ${collection.slug} ${id}`);
                                console.dir(doc);
                            }

                            if (doc._status === 'published') {
                                //Note: transaction commited later (see https://github.com/payloadcms/payload/blob/b73fc586b884a050b246afed7d5a032a5da3ee85/packages/payload/src/collections/operations/deleteByID.ts#L249), delete operation done before (see https://github.com/payloadcms/payload/blob/b73fc586b884a050b246afed7d5a032a5da3ee85/packages/payload/src/collections/operations/deleteByID.ts#L162)

                                //call hooks
                                const args: AfterPublishArgs = {
                                    action: 'deleted',
                                    doc
                                };

                                for (const hook of afterPublish) {
                                    const res = hook(args);

                                    if (res instanceof Promise) {
                                        await res;
                                    }
                                }
                            }

                            return doc;
                        }
                    ]
                }
            };
        });

        return <Config>{
            ...config,

            collections
        };
    };
}

Will perform more checks tomorrow and migrate our code to use this new custom hook.

[rolandsaven]

You are a Godsend. Thanks.

[cbratschi]

@DanRibbens @JarrodMFlesch I propose those non-breaking changes to Payload:

• add afterCommit hook
• add new boolean parameter revision to hooks which specifies whether the main document got updated or only the versions collection

afterCommit makes sure the data was successfully passed to the database and following queries will read exactly that written data.

The revision parameter defines pure revision updates done here for instance:

https://github.com/payloadcms/payload/blob/main/packages/payload/src/collections/operations/updateByID.ts#L356

The document in the main collection will not change during this operation. Many hooks can just skip those updates for instance for performance reasons or to reduce calls to external cloud services.

[tlmader]

This default behavior makes the afterChange hook much more unintuitive to use. The simplest workaround I've found is to store the change detection data into a separate field that is only updated if not draft/autosave, then compare previousDoc and doc using that field. Since draft/autosave is supported first-class, I think a previousPublishedDoc would be a good arg to include in the existing hooks.