Commit Graph

7656 Commits

Author SHA1 Message Date
chrislu
ec1e74a6e8 feat: Add Parquet logical types to mq_schema.proto
Added support for Parquet logical types in SeaweedFS message queue schema:
- TIMESTAMP: UTC timestamp in microseconds since epoch with timezone flag
- DATE: Date as days since Unix epoch (1970-01-01)
- DECIMAL: Arbitrary precision decimal with configurable precision/scale
- TIME: Time of day in microseconds since midnight

These types enable advanced analytics features:
- Time-based filtering and window functions
- Date arithmetic and year/month/day extraction
- High-precision numeric calculations
- Proper time zone handling for global deployments

Regenerated protobuf Go code with new scalar types and value messages.
2025-09-03 07:18:58 -07:00
chrislu
d60c542ecc feat: Replace pg_query_go with lightweight SQL parser (no CGO required)
- Remove github.com/pganalyze/pg_query_go/v6 dependency to avoid CGO requirement
- Implement lightweight SQL parser for basic SELECT, SHOW, and DDL statements
- Fix operator precedence in WHERE clause parsing (handle AND/OR before comparisons)
- Support INTEGER, FLOAT, and STRING literals in WHERE conditions
- All SQL engine tests passing with new parser
- PostgreSQL integration tests can now build without CGO

The lightweight parser handles the essential SQL features needed for the
SeaweedFS query engine while maintaining compatibility and avoiding CGO
dependencies that caused Docker build issues.
2025-09-03 07:11:18 -07:00
chrislu
4d9de40c5c fmt 2025-09-03 00:48:09 -07:00
chrislu
42661ac110 fix tests 2025-09-03 00:47:08 -07:00
chrislu
991247facf fix tests 2025-09-03 00:40:03 -07:00
chrislu
e3e369c264 change to pg_query_go 2025-09-03 00:10:47 -07:00
chrislu
ba4a8b91d5 fmt 2025-09-02 22:31:53 -07:00
chrislu
59d6806146 fix empty spaces and coercion 2025-09-02 22:30:52 -07:00
Chris Lu
f29dd385cc Merge branch 'master' into add-sql-querying 2025-09-02 22:14:21 -07:00
chrislu
3fa7670557 fix todo 2025-09-02 22:12:47 -07:00
chrislu
687c5d6bfd fix tests 2025-09-02 21:21:59 -07:00
chrislu
e14a316aeb use schema instead of inferred result types 2025-09-02 20:59:13 -07:00
chrislu
316d1cdda7 address some comments 2025-09-02 19:58:41 -07:00
chrislu
a7eb178cec Update engine.go 2025-09-02 18:37:31 -07:00
chrislu
60066a6a4c read broker, logs, and parquet files 2025-09-02 18:15:26 -07:00
chrislu
59ec4eb68a address comments 2025-09-02 17:37:52 -07:00
chrislu
d29f54e0be de-support alter table and drop table 2025-09-02 17:23:46 -07:00
chrislu
8e15fdf2c7 remove sample data 2025-09-02 17:18:19 -07:00
chrislu
f776a49322 avoid sample data 2025-09-02 17:15:13 -07:00
chrislu
ed7102df6e column name can be on left or right in where conditions 2025-09-02 17:05:31 -07:00
chrislu
900bd94456 address some comments 2025-09-02 16:54:34 -07:00
chrislu
e901abffd3 address comments 2025-09-02 15:40:38 -07:00
chrislu
ac83adc6ce fix versioning 2025-09-02 15:14:46 -07:00
chrislu
586a795b02 return fast on error 2025-09-02 14:09:38 -07:00
chrislu
1a2225c0f5 rename to "weed db" 2025-09-02 11:02:08 -07:00
chrislu
c9e093194d setup integration test for postgres 2025-09-02 10:51:37 -07:00
chrislu
8b4914be55 hook up seaweed sql engine 2025-09-02 08:42:36 -07:00
chrislu
5a8c31fcda Revert "add jdbc"
This reverts commit a6e48b7690.
2025-09-02 08:25:39 -07:00
chrislu
85306bb2b0 add postgres protocol 2025-09-02 08:25:29 -07:00
chrislu
a6e48b7690 add jdbc 2025-09-02 06:56:49 -07:00
chrislu
6fb88a8edb buffer start stored as 8 bytes 2025-09-02 00:57:48 -07:00
chrislu
61bacd23b0 parquet file can query messages in broker memory, if log files do not exist 2025-09-02 00:49:12 -07:00
chrislu
618cb89885 the parquet file should also remember the first buffer_start attribute from the sources 2025-09-02 00:42:37 -07:00
chrislu
db75742e37 explain with broker buffer 2025-09-02 00:35:52 -07:00
chrislu
de866bfd09 dedup with buffer start index 2025-09-02 00:26:35 -07:00
chrislu
e3a56d7c30 filter out already flushed messages 2025-09-02 00:10:07 -07:00
chrislu
467034c8c7 process buffer from brokers 2025-09-01 23:45:52 -07:00
chrislu
7ca3b59c44 save source buffer index start for log files 2025-09-01 23:33:35 -07:00
chrislu
f5ed25f755 fix decoding data 2025-09-01 22:17:17 -07:00
chrislu
99bfe95e51 detailed logs during explain. Fix bugs on reading live logs. 2025-09-01 22:16:46 -07:00
chrislu
c7a0b89067 fix after refactoring 2025-09-01 21:40:24 -07:00
chrislu
e385f0ce7d refactor 2025-09-01 20:19:59 -07:00
chrislu
61ad3c39ac add tests 2025-09-01 20:05:24 -07:00
chrislu
4214d765cf use mock for testing 2025-09-01 19:36:47 -07:00
chrislu
a3f6a5da27 skip 2025-09-01 18:55:14 -07:00
chrislu
dfd0897e49 improve tests 2025-09-01 18:52:22 -07:00
chrislu
7d88a81482 add tests 2025-09-01 18:00:55 -07:00
chrislu
eaa7136c92 explain the execution plan 2025-09-01 17:19:36 -07:00
chrislu
93a09f5da4 explain 2025-09-01 16:47:52 -07:00
chrislu
55cad6dc4a combine parquet results with live logs 2025-09-01 16:26:35 -07:00