Skip to content

App crash when events.js throws "socket hang up" even when I have error handle #1790

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
datvm opened this issue Jul 22, 2021 · 0 comments
Closed
Labels
investigate Under investigation and may be a bug.

Comments

@datvm
Copy link

datvm commented Jul 22, 2021

Describe the bug

So I usually receive this error while the script is running:

events.js:292
      throw er; // Unhandled 'error' event
      ^

Error: socket hang up
    at connResetException (internal/errors.js:607:14)
    at TLSSocket.socketOnEnd (_http_client.js:493:23)
    at TLSSocket.emit (events.js:327:22)
    at endReadableNT (internal/streams/readable.js:1327:12)
    at processTicksAndRejections (internal/process/task_queues.js:80:21)
Emitted 'error' event on ClientRequest instance at:
    at TLSSocket.socketOnEnd (_http_client.js:493:9)
    at TLSSocket.emit (events.js:327:22)
    at endReadableNT (internal/streams/readable.js:1327:12)
    at processTicksAndRejections (internal/process/task_queues.js:80:21) {
  code: 'ECONNRESET'
}

My app subscribe to the block event (provider.on("block", ...)). I followed #1053 to add reconnection ability but somehow the exception still went throw and the whole app crashes. I even added extra try/catch and on("error") handle (see below).

const EXPECTED_PONG_BACK = 15000
const KEEP_ALIVE_CHECK_INTERVAL = 7500

export class ConnectionService {

    public pro: ethers.providers.WebSocketProvider;
    public signer: ethers.Wallet;

    private blockSubscriptions: BlockCallback[] = [];

    private pingTimeout: NodeJS.Timeout = null;
    private keepAliveInterval: NodeJS.Timeout = null;

    constructor(
        private s: IScriptSetting,
        private l: LoggingService,
    ) {
        this.init();
    }

    init() {
        const [s, l] = [this.s, this.l];

        const pro = this.pro = new ethers.providers.WebSocketProvider(s.server);
        this.signer = new Wallet(s.accPri, pro);

        const ws = pro._websocket;

        ws.on("open", () => {
            l.info({
                message: "WebSocket connection opened",
                category: LogType.ScriptHealth
            });

            this.internalSubscribe();

            this.keepAliveInterval = setInterval(() => {
                ws.ping();

                this.pingTimeout = setTimeout(() => {
                    l.info({
                        message: "WebSocket Ping-Pong not received",
                        category: LogType.ScriptHealth,
                    });

                    ws.terminate();
                }, EXPECTED_PONG_BACK);
            }, KEEP_ALIVE_CHECK_INTERVAL);
        });

        ws.on("close", () => {
            this.resetConnection();
        });

        ws.on("pong", () => {
            clearTimeout(this.pingTimeout);
        });

        ws.on("error", (ex) => {
            l.error({
                message: "Connection Error",
                error: ex,
                category: LogType.ScriptHealth,
            });

            this.resetConnection();
        })
    }

    resetConnection() {
        this.l.info({
            message: "WebSocket connection closed",
            category: LogType.ScriptHealth
        });

        clearInterval(this.keepAliveInterval);
        clearTimeout(this.pingTimeout);

        this.init();
    }

    public subscribe(callback: BlockCallback): void {
        this.blockSubscriptions.push(callback);
    }

    private internalSubscribe() {
        try {
            this.pro.on("block", (blockNo: number) => {
                for (let sub of this.blockSubscriptions) {
                    sub(blockNo);
                }
            });
        } catch (e) {
            this.l.error({
                message: "Connection Error",
                error: e,
                category: LogType.ScriptHealth,
            });

            this.resetConnection();
        }
    }

}

export type BlockCallback = (blockNo: number) => any;

Environment:

NodeJS v14.16.1 on Windows 10 Pro and Windows Server 2019.

Node Client is latest stable Geth on Windows Server 2019.

Search Terms

WebSocket, reconnect, socket hang up

Please advice on how to fix this, and if possible is there a way that I can "simulate" a WS disconnect to test if my fix works? Thanks

@datvm datvm added the investigate Under investigation and may be a bug. label Jul 22, 2021
@datvm datvm closed this as completed Apr 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
investigate Under investigation and may be a bug.
Projects
None yet
Development

No branches or pull requests

1 participant