Apache ORC

The other Hadoop-era columnar file format, from the ORC v1 specification (T1). ORC — Optimized Row Columnar — came out of Hive, as a replacement for RCFile, and the improvement was type awareness: RCFile treated columns as binary blobs, ORC keeps the schema’s types and therefore can apply type-specific encodings (dictionary, bit packing, delta, run-length).

Structure

Header (ORC), body, tail. The body is divided into stripes of roughly 200 MB, each independently readable and containing index streams, data streams and a stripe footer; “rows never straddle stripe boundaries.” The tail holds stripe statistics, the file footer (stripe layout, schema, row counts, column statistics, writer implementation) and a postscript that is never compressed, never more than 256 bytes, and ends one byte before EOF — it tells a reader the compression codec and footer length, which is the bootstrap problem every self-describing format has to solve somehow.

Skipping is the point

ORC carries “light weight indexes that include the minimum and maximum values for each column in each set of 10,000 rows and the entire file,” plus optional bloom filters (Hive 1.2.0). Combined with column projection, that lets a reader “skip entire sets of rows that aren’t important for this query” — predicate pushdown implemented in the file rather than in the engine. Compression is per chunk (default 256 KB; zlib, Snappy, LZO, LZ4, ZSTD), so a reader can skip compressed data without decompressing it.

Against Parquet

Structurally these two are near-twins — row groups vs stripes, footers vs tails, per-chunk compression, min/max statistics, bloom filters — which is exactly why columnar-format-evaluation can benchmark them head to head, and why its verdict lands on both at once: the differences that matter are encoding defaults, not architecture. Where they diverge is lineage, and it shows in the tooling: parquet became the interchange format of the wider ecosystem, ORC stayed closest to Hive and its descendants.