add tag
kus
I have a question about date time and javascript. 

Some background: Github actions allows me to run arbitrary code but the catch is I must finish my work within six hours or get killed (my task, not me).

So I figured I will just add a const at the start of my file with the current time and then check if it has been five hours and thirty minutes before doing any intensive work and only do that work if it has NOT been five hours and thirty minutes. 

I think I have some fundamental flaw in my logic above because the action still keeps running for ~6 hours. Any ideas? 

In my code, I have const start_time = Date.now(); and then I have another Date.now() inside a function. I’m trying to stop puppeteer if the thing takes more than five and a half hours. The whole code is at [GitHub](https://github.com/scrapeaction/techmeme2/blob/53ee2db9b26c0e4add38fc603bd5ebcc11245054/puppeteer.js). 


```js
'use strict';
const puppeteer = require('puppeteer');

const start_time = Date.now();
// const total_allowed_time = (((5 * 60) + 30) * 60) * 1000;
const total_allowed_time = 1978643;

crawlPage("https://techmeme.com", "techmeme");
crawlPage("https://techmeme.com/river", "river");
crawlPage("https://techmeme.com/lb", "leaderboards");

function delay(time) {
    console.warn(`waiting for ${time / 1000} seconds...`);
    return new Promise(function (resolve) {
        setTimeout(resolve, time)
    });
}

function crawlPage(url, prefix) {
    (async () => {

        const args = [
            "--disable-setuid-sandbox",
            "--no-sandbox",
            "--blink-settings=imagesEnabled=true",
        ];
        const options = {
            args,
            headless: true,
            ignoreHTTPSErrors: true
        };

        const browser = await puppeteer.launch(options);
        const page = await browser.newPage();
        await page.setViewport({
            width: 1920,
            height: 1080
        });

        await page.goto(url, {
            waitUntil: 'networkidle0',
            timeout: 0
        });

        await page.screenshot({
            path: `screenshots/${prefix}.png`,
            fullPage: true
        });
        await page.screenshot({
            path: `screenshots/${prefix}-fold.png`,
            fullPage: false
        });

        const addresses = await page.$$eval('a', as => as.map(a => a.href));
        const padding = addresses.length % 10;
        for (let i = 0; i < addresses.length; i++) {
            try {
                if ((Date.now() - start_time < total_allowed_time)
                    && addresses[i].startsWith("http") === true) {
                    console.log(`Now serving ${i} of ${addresses.length}: ${addresses[i]}`);
                    await page.goto(addresses[i], { waitUntil: "networkidle0", timeout: 0 });
                    
                    await delay(1000);

                    const watchDog = page.waitForFunction(() => 'window.status === "ready"', { timeout: 0 });
                    await watchDog;

                    await page.screenshot({
                        path: `screenshots/${prefix}-${i.toString().padStart(padding, '0')}.png`,
                        fullPage: true
                    });

                    await page.screenshot({
                        path: `screenshots/${prefix}-${i.toString().padStart(padding, '0')}-fold.png`,
                        fullPage: false
                    });
                }
                else {
                    console.log(`Skip serving ${i} of ${addresses.length}: ${addresses[i]}`);
                }
            } catch (error) {
                console.error(error);
            } finally {
                console.log(`Finished with ${i} of ${addresses.length}: ${addresses[i]}`);
                console.log(`Time now: ${Date.now()}. Start time: ${start_time}`);
                console.warn(`Time elapsed: ${ (Date.now() - start_time) / (1000 * 60) } minutes`); 
                console.warn(`Time elapsed: ${ (Date.now() - start_time) / (1000 * 60 * 60) } hours`); 
            };
        }

        await page.close();
        await browser.close();

    })().catch((error) => {
        console.error(error);
    });

}
```

I tried to create an sccccce (can never remember how many c's are there in sccce) but that didn't help me at all.


The GH Action [logs](https://github.com/scrapeaction/techmeme2/runs/1287821699):
```
Run node puppeteer.js
Skip serving 0 of 315: javascript:nwcbe.checked=!nwcbe.checked;xnwcb();
Time elapsed: 0.12678333333333333 minutes
Time elapsed: 0.0021130555555555555 hours
Finished with 0 of 315: javascript:nwcbe.checked=!nwcbe.checked;xnwcb();
Time now: 1603296192767. Start time: 1603296185160
Now serving 1 of 315: https://www.techmeme.com/feed.xml
Skip serving 0 of 330: javascript:nwcbe.checked=!nwcbe.checked;xnwcb();
Finished with 0 of 330: javascript:nwcbe.checked=!nwcbe.checked;xnwcb();
Time elapsed: 0.12728333333333333 minutes
Time now: 1603296192797. Start time: 1603296185160
Time elapsed: 0.002121388888888889 hours
Now serving 1 of 330: https://www.techmeme.com/feed.xml
Skip serving 0 of 1619: javascript:nwcbe.checked=!nwcbe.checked;xnwcb();
...
...
...
Time now: 1603298174023. Start time: 1603296185160
Skip serving 328 of 330: https://techmeme.com/events
Finished with 328 of 330: https://techmeme.com/events
Time now: 1603298174023. Start time: 1603296185160
Skip serving 329 of 330: http://eepurl.com/dt9xC1
Finished with 329 of 330: http://eepurl.com/dt9xC1
Time now: 1603298174024. Start time: 1603296185160
Error: The operation was canceled.
```
Top Answer
Pax
The issue is that your script was not able to close your browsers properly. 

Some observations:
1. The last few lines of your logs show that URLs _are_ being skipped.
2. The first time that a URL was skipped was because the script has been running for ~32 minutes (the `total_allowed_time`)
3. The last couple of lines in the logs show that it finished looping through the 330 addresses (it says 329 of 330, but you're actually printing the index "329")
4. Immediately after looping, the action encountered an error that somehow kept the script from continuing to loop through the 1619 and 315 addresses from the other 2 URLs.

Based on 1 & 2, we can safely say that your understanding of dates in JS fine.

Based on 3 & 4, I'm thinking the browser/s (launched by puppeteer) are still open. GH will see the script as still "running" and will wait for it to end until for however long GH allows.
  
The next thing I would try to figure out is where this  "Error: The operation was canceled." coming from. And also add logs after "browser.close()" to see whether the browsers were closed successfully.

Enter question or answer id or url (and optionally further answer ids/urls from the same question) from

Separate each id/url with a space. No need to list your own answers; they will be imported automatically.