Puppeteer - scroll down until you can't anymore - script failing

Question

I want to automate deleting my ChatGPT chats because I have about 244 of them, and I’m not going to do that manually. What I want to do is scroll down to the bottom until there are no more chat items, then delete them from last to first. The deletion part works, but I’m having some issues with the scrolling part.

console.log("scrollToBottom has been called");

await page.evaluate(async () => {
    const delay = 10000;
    const wait = (ms) => new Promise(res => setTimeout(res, ms));
    const sidebar = document.querySelector('#stage-slideover-sidebar');

    const count = () => document.querySelectorAll('#history aside a').length;

    const scrollDown = async () => {
        const lastChild = document.querySelector('#history aside a:last-child');
        if (lastChild) {
            lastChild.scrollIntoView({ behavior: 'smooth', block: 'end', inline: 'end' });
        }
    }

    let preCount = 0;
    let postCount = 0;
    let attempts = 0;
    do {
        preCount = count(); 
        await scrollDown();
        await wait(delay);
        postCount = count(); 
        console.log("preCount", preCount, "postCount", postCount, "attempts", attempts);
        if (postCount === preCount) {
            attempts++;
        } else {
            attempts = 0;
        }
    }  while (attempts < 10);

    console.log("Reached bottom. Total items:", postCount);

    // await wait(delay);
});

This works better than when I set the delay to 1, 2, or 3 seconds and the attempts to 3. When I use this, it stops loading at 84. However, the issue I have with a 10-second delay and 10 attempts is that I run into this error after everything has loaded.

    #error = new Errors_js_1.ProtocolError();
             ^

ProtocolError: Runtime.callFunctionOn timed out. Increase the 'protocolTimeout' setting in launch/connect calls for a higher timeout if needed.
    at <instance_members_initializer> (/Users/pc/WebstormProjects/puppeteer/node_modules/puppeteer-core/lib/cjs/puppeteer/common/CallbackRegistry.js:102:14)
    at new Callback (/Users/pc/WebstormProjects/puppeteer/node_modules/puppeteer-core/lib/cjs/puppeteer/common/CallbackRegistry.js:106:16)
    at CallbackRegistry.create (/Users/pc/WebstormProjects/puppeteer/node_modules/puppeteer-core/lib/cjs/puppeteer/common/CallbackRegistry.js:24:26)
    at Connection._rawSend (/Users/pc/WebstormProjects/puppeteer/node_modules/puppeteer-core/lib/cjs/puppeteer/cdp/Connection.js:99:26)
    at CdpCDPSession.send (/Users/pc/WebstormProjects/puppeteer/node_modules/puppeteer-core/lib/cjs/puppeteer/cdp/CdpSession.js:73:33)
    at #evaluate 
(/Users/pc/WebstormProjects/puppeteer/node_modules/puppeteer-core/lib/cjs/puppeteer/cdp/ExecutionContext.js:363:50)
    at ExecutionContext.evaluate (/Users/pc/WebstormProjects/puppeteer/node_modules/puppeteer-core/lib/cjs/puppeteer/cdp/ExecutionContext.js:277:36)
    at IsolatedWorld.evaluate (/Users/pc/WebstormProjects/puppeteer/node_modules/puppeteer-core/lib/cjs/puppeteer/cdp/IsolatedWorld.js:100:30)
    at CdpFrame.evaluate (/Users/pc/WebstormProjects/puppeteer/node_modules/puppeteer-core/lib/cjs/puppeteer/api/Frame.js:364:43)
    at CdpFrame.<anonymous> (/Users/pc/WebstormProjects/puppeteer/node_modules/puppeteer-core/lib/cjs/puppeteer/util/decorators.js:109:27)

Node.js v22.19.0

How best can I approach this?

ggorlen · Accepted Answer · 2025-09-12 19:58:25Z

What I want to do is scroll down to the bottom until there are no more chat items, then delete them from last to first

Why delete from last to first? When you delete the top conversation, then the new top conversation is the one below it. Therefore, you can write a loop to delete the top conversation repeatedly until there are no conversations left, without scrolling. Scrolling is flaky, so we want to avoid it at all costs.

Another option is to hit this endpoint in a loop to page through all of your conversations: https://chatgpt.com/backend-api/conversations?offset=0&limit=50&order=updated&is_archived=false&is_starred=false. For each conversation, call PATCH https://chatgpt.com/backend-api/conversation/:conversation-id. The full payloads can be obtained from the network tab in devtools.

Or you can take a hybrid approach, where you collect the id from the first conversation, issue a deletion request with fetch and repeat until no conversations remain. I ran this without Puppeteer in the browser devtools and it deleted a bunch of conversations. I didn't run it all the way to completion since I don't want to clear my history out. You can consider it a proof of concept.

(async () => {
  for (;;) {
    const convo = document.querySelector("a[href*='/c/']");

    if (!convo) {
      break;
    }

    const id = convo.href.split("/").pop();

    // copy this from the dev tools when you delete a convo, but keep my URL with `${id}`
    const response = await fetch(
      `https://chatgpt.com/backend-api/conversation/${id}`,
      {
        credentials: "include",
        referrer: "https://chatgpt.com/",
        body: '{"is_visible":false}',
        method: "PATCH",
        mode: "cors",
        headers: {
          "User-Agent":
            "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:142.0) Gecko/20100101 Firefox/142.0",
          "Accept": "*/*",
          "Accept-Language": "en-US,en;q=0.5",
          "OAI-Language": "en-US",
          "OAI-Device-Id":
            "b974e911-795e-443a-b90d-0a8ec56bd5cb",
          "OAI-Client-Version":
            "prod-716e5d4247cec821c6247f75b3bd23aca98bcfd7",
          "Authorization": "Bearer eyJhbGciOiJSUzI...",
          "Content-Type": "application/json",
          "Sec-Fetch-Dest": "empty",
          "Sec-Fetch-Mode": "cors",
          "Sec-Fetch-Site": "same-origin",
          "Priority": "u=0",
        },
      }
    );

    if (!res.ok) {
      throw Error(await res.text());
    }

    convo.remove();
    await new Promise(r => setTimeout(r, 5000)); // avoid rate limits
  }
})();

But as with many things in life, digging around a bit reveals a simple way to do this. You can click a button in settings to delete all of your chats:

Realized now, chatgpt has a function for it, thanks. Why bottom to top was because top still has my most recent conversation. If I were still using one it will go last.

Collectives™ on Stack Overflow

Puppeteer - scroll down until you can't anymore - script failing

1 Answer 1

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related