Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Cloudflare workers #337

Open
vedantroy opened this issue Apr 28, 2022 · 15 comments · May be fixed by #867
Open

Support Cloudflare workers #337

vedantroy opened this issue Apr 28, 2022 · 15 comments · May be fixed by #867

Comments

@vedantroy
Copy link

It would be nice if the Edge DB JS client supported Cloudflare workers: https://workers.cloudflare.com/

To do this, any references to unsupported node polyfills would need to be removed.
It seems like there is only one? (Buffer)

@vedantroy vedantroy changed the title Cloudflare workers Support Cloudflare workers Apr 28, 2022
@elprans elprans transferred this issue from geldata/gel Apr 28, 2022
@colinhacks
Copy link
Contributor

I believe this would require a ground-up rewrite, since we rely on a ton of Node.js APIs from crypto, fs, path, os, net, tls, readline, and stream. Unless I'm missing something? cc @tailhook

@elprans
Copy link
Member

elprans commented May 4, 2022

What about Deno? I've read somewhere that Deno is explicitly compatible with Cloudflare workers.

@tailhook
Copy link
Contributor

tailhook commented May 4, 2022

I believe this would require a ground-up rewrite, since we rely on a ton of Node.js APIs from crypto, fs, path, os, net, tls, readline, and stream. Unless I'm missing something? cc @tailhook

I think most of that is possible to get rid of (fs and path are probably by disabling that feature), but the biggest issue is raw network sockets. I think CloudFlare workers don't support them. So we need some WebSocket mapping of the protocol, which I think we didn't implement yet.

@elprans
Copy link
Member

elprans commented May 4, 2022

We actually already implement protocol tunneling over the Fetch API. The only thing that isn't supported is transaction blocks.

@haikyuu
Copy link

haikyuu commented Oct 21, 2022

Is transaction support coming in the near future (4-8 months) for cf workers?

@elprans
Copy link
Member

elprans commented Oct 21, 2022

Transactions are supported in edgedb+http, just only the all-at-once kind. I.e you can send a bunch of statements together and they'll be executed atomically. But you can't start a transaction and keep it open. The difficulty of implementing open-ended transactions over a stateless protocol like HTTP is that you need sticky sessions, which introduces lots of complexity to the stack.

@haikyuu, do you have an use case example that can't be done as a single db roundtrip?

@haikyuu
Copy link

haikyuu commented Oct 21, 2022

@haikyuu, do you have an use case example that can't be done as a single db roundtrip?

No that's probably plenty enough for the foreseeable future 😅

@scotttrinh
Copy link
Collaborator

Related to #387

@PastelStoic
Copy link
Contributor

From what I understand of the HttpClient, both this and #387 can be closed.

@bdematt
Copy link

bdematt commented Jan 5, 2024

@PastelStoic Is this resolved? I'm trying to call createHttpClient in a Cloudflare Function and still getting the error ReferenceError: process is not defined.

@TreeOfLearning
Copy link

TreeOfLearning commented Jan 15, 2024

@PastelStoic Is this resolved? I'm trying to call createHttpClient in a Cloudflare Function and still getting the error ReferenceError: process is not defined.

Yep - also running into this, specifically from a call to getEnv in parseConnectDsnAndArgs; so my guess is this is not resolved :/ Sample:

const e = edgedb.createHttpClient({
    dsn: env.EDGEDB_DSN,
    tlsSecurity: "insecure",
});

const res = await e.querySingle<number>("SELECT 1");

@scotttrinh scotttrinh linked a pull request Feb 15, 2024 that will close this issue
@bitnom
Copy link

bitnom commented Jul 12, 2024

wondering about this before deciding to use edgedb

@bitnom
Copy link

bitnom commented Jul 12, 2024

Raw sockets are now supported via the connect() method. See TCP Sockets docs

@bitnom
Copy link

bitnom commented Jul 12, 2024

Maybe something like this. Idk what the filesystem stuff is even used for

// adapter.cloudflare.ts

import { connect } from 'cloudflare:sockets';
export { Buffer } from 'node:buffer';

// Stub implementations for incompatible modules
export const path = {};
export const process = {};
export const util = {};
export const fs = {};

// File system related functions (not supported in Workers)
export async function readFileUtf8(...pathParts: string[]): Promise<string> {
  throw new Error('File system operations are not supported in Cloudflare Workers');
}

export function hasFSReadPermission(): boolean {
  return false;
}

export async function readDir(path: string) {
  return [];
}

export async function walk(
  path: string,
  params?: { match?: RegExp[]; skip?: RegExp[] },
) {
  return [];
}

export async function exists(fn: string | URL): Promise<boolean> {
  return false;
}

// Crypto and environment-related functions
export function hashSHA1toHex(msg: string): string {
  const encoder = new TextEncoder();
  const data = encoder.encode(msg);
  const hashBuffer = crypto.subtle.digest('SHA-1', data);
  return Array.from(new Uint8Array(hashBuffer))
    .map(b => b.toString(16).padStart(2, '0'))
    .join('');
}

export function homeDir(): string {
  throw new Error('Home directory is not available in Cloudflare Workers');
}

export async function input(message = "", _params?: { silent?: boolean }) {
  throw new Error('User input is not supported in Cloudflare Workers');
}

// Networking-related classes and functions
export namespace net {
  export function createConnection(port: number, hostname?: string): Socket;
  export function createConnection(unixpath: string): Socket;
  export function createConnection(
    port: number | string,
    hostname?: string,
  ): Socket {
    if (typeof port === 'string') {
      throw new Error('Unix socket connections are not supported in Cloudflare Workers');
    }
    return new Socket(port, hostname);
  }

  export function isIP(input: string): 0 | 4 | 6 {
    if (/^(\d{1,3}\.){3}\d{1,3}$/.test(input)) return 4;
    if (/^[0-9a-fA-F:]+$/.test(input)) return 6;
    return 0;
  }

  export class Socket extends EventTarget {
    private connection: Promise<any>;

    constructor(port: number, hostname?: string) {
      super();
      this.connection = connect({ hostname: hostname || 'localhost', port });
    }

    async write(data: Uint8Array) {
      const socket = await this.connection;
      const writer = socket.writable.getWriter();
      await writer.write(data);
      writer.releaseLock();
    }

    setNoDelay() {}
    unref() { return this; }
    ref() { return this; }
    pause() {}
    resume() {}
    destroy(error?: Error) {
      if (error) throw error;
    }
  }
}

// TLS-related functions
export namespace tls {
  export function connect(options: tls.ConnectionOptions): tls.TLSSocket {
    return new TLSSocket(options);
  }

  export function checkServerIdentity(
    _hostname: string,
    _cert: object,
  ): Error | undefined {
    return undefined;
  }

  export interface ConnectionOptions {
    host?: string;
    port?: number;
    ALPNProtocols?: string[];
    ca?: string | string[];
    checkServerIdentity?: (a: string, b: any) => Error | undefined;
    rejectUnauthorized?: boolean;
    servername?: string;
  }

  export class TLSSocket extends net.Socket {
    private _alpnProtocol: string | null = null;

    constructor(options: ConnectionOptions) {
      super(options.port!, options.host);
    }

    get alpnProtocol(): string | false {
      return this._alpnProtocol ?? false;
    }
  }
}

// Utility functions
export function exit(code?: number) {
  throw new Error('Cannot exit in Cloudflare Workers environment');
}

export function srcDir() {
  return '/';
}

@scotttrinh
Copy link
Collaborator

@bitnom

Thanks for the exploration here! I think we are actually a lot closer to supporting Cloudflare workers without this custom adapter using the node_compat: https://github.com/edgedb/edgedb-examples/tree/main/cloudflare-workers

But knowing there is now a direct TCP API is helpful, and thanks for sketching out an initial implementation here, that'll be helpful. PRs welcome, but I know this isn't a trivial part of the code base to hack on.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

10 participants