Syncing migrated self-managed tables

PostHog no longer links tables that read a bucket directly. Eligible existing self-managed tables now have a managed source and a schema that is turned off. The table keeps working and keeps its name. It costs nothing until someone turns syncing on. Turning syncing on starts billing, so the user must agree first.

What happened

• PostHog made one source for each bucket and key prefix. Tables that need different access rules get different sources.
• Each table got one schema with should_sync set to false. The schema reads the same files that the table read.
• The table keeps its id and name. Queries, views and insights that use it keep working.
• Until the first sync, ClickHouse still reads the user's bucket on every query. There is no charge for this.
• The first successful sync "adopts" the table. PostHog copies the data into its own storage and points the same table row at the copy. After that, the table is a normal imported table and is billed like one.
• Source descriptions show the origin: "Self-managed tables in " for bucket sources and "Uploaded file for " for file upload sources.

Available tools