Brian Hulette

I had a look this raw arrow binary file is ~45MB, almost reaching the limit (50MB) host by gist, had a question from the forum, not sure if you're on the forum also? my plain text json is just growing over the 50MB limit unable to host on gist, not sure if can convert to use arrow binary format and continue to use gist for some months? So does arrow have nested data types and some what compression?     Thanks;
https://talk.observablehq.com/t/whats-best-data-format-for-raw-data-file-size-at-50-300mb-and-if-not-gist-where-is-a-better-place-to-host-for-free/3869

$ curl -I https://gist.githubusercontent.com/TheNeuralBit/64d8cc13050c9b5743281dcf66059de5/raw/c146baf28a8e78cfe982c6ab5015207c4cbd84e3/scrabble.arrow
HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 46978004           <= this is ~45MB

does JS bindings have the same `read_json` as in Python bindings? to convert a line-delimited json file to arrow?

https://arrow.apache.org/docs/python/json.html

the IPC formats link is a 404. Apparently now integrated with https://arrow.apache.org/docs/format/Columnar.html though https://arrow.apache.org/docs/format/Flight.html may also be of interest

Jacob Rus

Thanks Jacob, updated this to point to https://arrow.apache.org/docs/format/Columnar.html#serialization-and-interprocess-communication-ipc