Benjamin Schmidt

I tend to keep up to date with IPEDS and related datasets from the Department of Education

Higher Education Datasets

I think this can just be `SELECT DISTINCT Discipline FROM "fields.parquet"`.

Dominik Moritz

Yeah my SQL is from MySQL about ten years ago, so it's rusty in multiple ways!

these are all red for me.

'RuntimeError: Catalog Error: Table with name characteristics@2 does not exist!
Did you mean "characteristics"?
LINE 1: ...l_filter AS SELECT UNITID, INSTNM FROM "characteristics@2" WHERE TRUE AND "STA...
                                    '

Tom Larkworthy

Thanks for the heads up. Looks like my personal copy had some changes not reflected in the published branch. It should hopefully work now?

Do you have an origin for the discussion about access-control-allow-headers? Range would be easy enough to add add to the cors headers, but I’m not entirely sure how good cloudfront/s3 support is (it seems to exist but I’ve never stressed it).

Visnu Pitiyanuvath

I verified ranged fetches of file attachments don't work on published notebooks, but you could attempt them on private notebooks/attachments for now (those don't cross a CORS boundary and seem to work for me).

Thanks for checking Pasted a link to the github discussion here in the block, https://github.com/duckdb/duckdb-wasm/discussions/356. 
Dominick Moritz links from there to an issue he filed on observable at Mike Bostock's request. https://github.com/observablehq/feedback/issues/283

That's great if it works on local ones.

Looking at it a little more, I think this specific notebook probably would benefit less than others from the partial range approach anyway, because it does a full table scan relatively early.

aha, thank you for the context I was missing. will see how easy this is…

It would be great if we could do range requests against files on e.g. GitHub. That wouldn't require support for range requests in cloud front yet, right? As a second step, support range requests over file attachments would be sweet as well.

Let's move discussion to the feedback issue (https://github.com/observablehq/feedback/issues/283). I updated there on progress and could use some feedback.

Ranges should now be supported when fetching from FileAttachment URLs, at least for binary files.

Is this comment up to date? The discussion above and the issue https://github.com/observablehq/feedback/issues/283 seem to say it's now the duckdb client that would need some kind of update?

I think DuckDB now uses range requests. Do you see that it doesn't work? (FYI, I fixed a typo in my comment on GitHub).

Yeah I think it's fixed now! Thanks. Changed my comment.

you could make the height dynamic, like `2.5 * majors.toArray().length` instead of fixed `2000`, to adjust the gap between rows for varying number of records.