2c:null 2d:["$","title",null,{"children":"ISO Base Media File Format | benjamintoofer | Observable"}] 32:T9c1,

Fragmented MP4 Structure

NOTE: This is not the complete list of boxes available in the specification. These are just the most common ones you may encounter with adjustable bitrate streaming. If you'd like to view the complete list, you may view in the ISO/IEC 14496-12 specification. I will go over in detail the following list.

Initialization Segment Structure

ftyp

moov

mvhd
trak
- tkhd
- edts
  - elst
- mdia
  - mdhd
  - hdlr
  - minf
    - vmhd
    - smhd
    - hmhd
    - nmhd
    - dinf
      - dref
    - stbl
      - stsd
      - stsc
      - stsz
      - stco
      - sdtp

Media Segment Structure

styp

moof

mfhd
traf
- tfhd
- trun
- tfdt

mdat

33:Tcbd,md` ## Box --- ### Description > This is the basic object structure. All boxes start with header data which gives you the **size** and **type** ### Properties > **Size**: Includes the box's type and size headers, fields, and other contained boxes
>**Type**: A uuid that identifies the box type ### Class View > \`\`\`javascript class Box { size; // 32 bit unsigned type; // 32 bit unsigned } \`\`\` **NOTE:** 32 bit size and type are compact. There can be an extended 64 bit size and type ### Diagram >

**box1** is the top level box and a container box. The header is \`40|box1\`. This size comes first and indicates that the box is **40** bytes big so eveything contained within it is should be **32** bytes (The additional **8** are deciated to the **4** byte for the size and **4** byte for the box type)
**box2** is also a container box. The header is \`32|box2\`. The box is **24** bytes big (The additional **8** are deciated to the **4** byte for the size and **4** byte for the box type)
**box3** and **box4** contain data. Each box is **12** bytes big. **8** of the bytes are for the header and the rest is the data.

`34:T128b,md` ## Full Box --- ### Description > This is an extention to the \`Box\`. Fullboxes contain an additional property called **version** and **flags** ### Properties > **Version**: Is an integer that specifies the version of this format of the box. Usually a **0** or a **1**. Different versions signal that the box's data is parsed differently
>**Flags**: Is a map of flags ### Class View > \`\`\`javascript class FullBox extends Box { version; // 8 unsigned bit flags; // 24 unsigned bit } \`\`\` ### Diagram `35:T1f50,md` ## File Type Box: \`ftyp\` --- ### Description > The **File Type Box** comes in the in the Audio and Video intialization segments. It is the heighest level box in these initialization segments (mp4s). The information in the \`ftyp\` have the inital information to tell teh video element how to decode the incoming segments. > There can only be one \`ftyp\` box per file and it is mandatory for the intialization segments. ### Properties > **MajorBrand**: This specifies to the video player what specifications it should expect from the files. Meaning what boxes to expect when reading the mp4 files
>**MinorBrand**: This is informative only. It does not require the files to conform to the standard's of the \`minorBrand\` but it can allow a more precise version of the \`majorBrand\`'s specifications
>**CompatibleBrands**: This tells the player that there are more brands. If the player is able to support the other brands it will play those aspects of the file (The other boxes that the compatible brand supports) ### Class View > \`\`\`javascript class FileTypeBox extends Box { majorBrand; // 32 unsigned bit minorBrand; // 32 unsigned bit compatibleBrands; // 32 unsigned bit [] } \`\`\` **NOTE**: Since it only extends box, it does not have the version and flag fields ### Example > #### Explanation >> **Size**: 28 - This means that the box is 28 bytes big
**Type**: \`ftyp\` - Specifying this box is of the type \`ftyp\`
**MajorBrand**: \`dash\` - The video player must accept \`dash\` file formats **MinorBrand**: 0 - No minor brand was specified
**CompatibleBrands** [iso6, avc1, mp41] - these are the other compatible file formats that are acceptable `36:Td63,md` ## Movie Header Box: \`mvhd\` --- ### Description > This box is inside the \`moov\` box since it constains metadata about the entir presentation. It contains information such as creation and modification time of the presentation. It tells the video player the total duration, the timescale,the initial play rate, and the intial volume. ### Properties > **Version**: This will wither be a **0** or **1**. This signals if some of teh fields are supposed to parsed differently. So if it is a **1** then \`creationTime\`, \`modificationTime\`, and \`duration\` are 64 bit values, otherwise they will be 32 bit values
**CreationTime**: The timestamp of when this presentation was created
**ModificationTime**: The timestamp of when this presentation was last modified
**Timescale**: How the time is scaled within this presentation. >> #### Example: If the timescale is **90000** then a duration of a segment in the presentation such as **180180** can be converted to seconds like so
180180/90000 = 2.002 seconds >**Duration**: How long the presentation is.
**Rate**: Indicates the preferred rate to play the presentation. The value is a fixed point 16.16 number.
>> #### Example: \`rate\` = 0x0100
\`rate\` = 0b00000000000000010000000000000000
\`rate\` = 0b0000000000000001.0000000000000000
\`rate\` = 1.0
This means that the \`rate\` is 1.0 in decimal value > **Volume**: Indicates the preferred playback volume. The value is a fixed point 8.8 number.
>> #### Example: \`volume\` = 0x00010000
\`volume\` = 0b0000000100000000
\`volume\` = 0b0000000100000000
\`volume\` = 1.0
This means that the \`volume\` is 1.0 in decimal value > **Matrix**: This is a transformation matrix, more specifically it is a [UVW Mapping](https://en.wikipedia.org/wiki/UVW_mapping). This is shows how to map points from one coordinate system to another. Traditionally it is an identity matrix.
>> #### Example: USE CANVAS TO DRAW MATRIX EXAMPLE > **NextTrackID**: This is a non-zero value that indicates what the next track is to be added to the presentation.

### Class View > \`\`\`javascript class MVHD extends FullBox { version; // 8 unsigned bit flags; // 24 unsigned bit creationTime; // 32 or 64 unsigned bit modificationTime; // 32 or 64 unsigned bit timescale; // 32 unsigned bit duration; // 32 or 64 unsigned bit rate; // 32 unsigned bit volume; // 16 unsigned bit __reserved__ // 16 bit __reserved__ // 32 bit [2] matrix; // 32 unsigned bit [9] (a 3 X 3 matrix) __pre_defined__ // 32 bit [6] nextTrackID // 32 unsigned bit } \`\`\` `37:Ta97,md` ## Track Header Box: \`tkhd\` --- ### Description > This box is placed within the \`trak\` box (Track Box) and there can only be one. It is mandatory and contains the metadata that describes the charactersitics of a single track. ### Properties > **Version**: This will wither be a **0** or **1**. This signals if some of the fields are supposed to parsed differently. So if it is a **1** then \`creationTime\`, \`modificationTime\`, and \`duration\` are 64 bit values, otherwise they will be 32 bit values
**CreationTime**: The timestamp of when this presentation was created
**ModificationTime**: The timestamp of when this presentation was last modified
**Duration**: How long the presentation is.
**Rate**: Indicates the preferred rate to play the presentation. The value is a fixed point 16.16 number.
>> #### Example: \`rate\` = 0x0100
\`rate\` = 0b00000000000000010000000000000000
\`rate\` = 0b0000000000000001.0000000000000000
\`rate\` = 1.0
This means that the \`rate\` is 1.0 in decimal value > **Volume**: Indicates the preferred playback volume. The value is a fixed point 8.8 number.
>> #### Example: \`volume\` = 0x00010000
\`volume\` = 0b0000000100000000
\`volume\` = 0b0000000100000000
\`volume\` = 1.0
This means that the \`volume\` is 1.0 in decimal value >**Matrix**: This is a transformation matrix, more specifically it is a [UVW Mapping](https://en.wikipedia.org/wiki/UVW_mapping). This is shows how to map points from one coordinate system to another. Traditionally it is an identity matrix.
### Class View > \`\`\`javascript class TKHD extends FullBox { version // 8 unsigned bit flags // 24 unsigned bit creationTime // 32 or 64 unsigned bit modificationTime // 32 or 64 unsigned bit trackID // 32 unsigned bit __reserved__ // 32 bit duration // 32 or 64 unsigned bit __reserved__[2] // 2 32 bit layer // 16 unsingned bit alternateGroup // 16 unsigned bit volume // 16 float bit __reserved__ // 16 bit matrix[9] // 3 X 3 32 float bit width // 32 unsigned bit height // 32 unsigned bit } \`\`\` `38:T5dd,md` ## Track Extends Box: \`trex\` --- ### Description > This box is placed within the \`mvex\` box (Movie Extends Box). This sets default values for for the movie fragments which ends up saving space for each \`traf\` box (Track Fragment Box). The header boxes (\`tfhd\`) for the Track Fragment Boxes have optional fields for the default values listed here but are empty if set in this box. ### Properties > **TrackID**: Identifies the track; this shall be the track ID of a track in the Movie Box.
**DefaultSampleDescriptionIndex**: Sets up the default value used in the track fragments.
**DefaultSampleDuration**: Sets up the default value used in the track fragments.
**DefaultSampleSize**: Sets up the default value used in the track fragments.
**DefaultSampleFlags**: Sets up the default value used in the track fragments.
### Class View > \`\`\`javascript class TREX extends FullBox { // Version and Flags fileds are 0 trackID // 32 unsigned bit defaultSampleDescriptionIndex // 32 unsigned bit defaultSampleDuration // 32 unsigned bit defaultSampleSize // 32 unsigned bit defaultSampleFlags // 32 unsigned bit } \`\`\` `39:Ta84,md` ## Track Fragment Header Box: \`tfhd\` --- ### Description > There can be zero or more fragments to each track and each track fragment can have multple contiguous runs of samples (can think of them as frames). This box sets up the information for the run of samples. ### Properties > **TFFlags**: All the data for tvideo and audio are contained in this box. It holds data for multiple frames.
>> #### Options: **base-data-offset-present**
0x000001
0b000000000000000001
Indicates if the BaseDataOffset field is present. The BaseDataOffset field tells the Track Run (\`trun\`) the offset starting from the beginning (\`moof\` + offset). If this flag isn't set, then position of the first byte is right after the \`moof\` header

**sample-description-index-present**
0x000002
0b000000000000000010
** TODO **

**default-sample-duration-present**
0x000008
0b000000000000001000
** TODO **

**default-sample-size-present**
0x000010
0b000000000000010000
** TODO **

**default-sample-flags-present**
0x000020
0b000000000000100000
** TODO **

**duration-is-empty**
0x010000
0b010000000000000000
** TODO **

**default-base-is-moof**
0x020000
0b100000000000000000
** TODO **

**TrackID**: All the data for tvideo and audio are contained in this box. It holds data for multiple frames.

**OPTIONAL FIELDS**
**BaseDataOffset**: All the data for tvideo and audio are contained in this box. It holds data for multiple frames.
**SampleDescriptionIndex**: All the data for tvideo and audio are contained in this box. It holds data for multiple frames.
**DefaultSampleDuration**: All the data for tvideo and audio are contained in this box. It holds data for multiple frames. ### Class View \`\`\`javascript class TrackFragmentHeaderBox extends FullBox { version // 8 unsigned bit (Should always be 0) flags // 24 unsigned bit trackID // 32 unsigned bit // OPTIONAL FIELDS baseDataOffset // 64 unsinged bit sampleDescriptionIndex // 32 unsigned bit defaultSampleDuration // 32 unsigned bit defaultSampleSize // 32 unsigned bit defaultSampleFlags // 32 unsigned bit } \`\`\` `3a:[] 30:{"id":"9e8b71e047be866b","slug":"iso-base-media-file-format","title":"ISO Base Media File Format","thumbnail":"0f79f4792ccde9aba4cd71e4b3c27239458d2a5277b932270a3751049f61cab5","access_level":"public","create_time":"$D2018-10-02T02:09:42.935Z","update_time":"$D2021-05-22T03:58:36.811Z","view_time":null,"first_public_time":"$D2019-11-13T19:54:59.911Z","has_unpublished_edits":null,"fork":false,"stars":1,"starred":false,"owner":{"type":"team","id":"28c8f10e400d2666","login":"benjamintoofer","name":"","avatar_url":"https://avatars-next.observableusercontent.com/e301ce5d3ad1a42b0de3f694a65c97ff71cd5e2efec0e0b0180bb068f25d2e39","tier":"starter_2024"},"version":617,"publish_version":617,"role":null,"authored":false,"authors":[{"type":"individual","id":"da05e359859db73e","login":"benjamintoofer","name":"","avatar_url":"https://avatars-next.observableusercontent.com/e301ce5d3ad1a42b0de3f694a65c97ff71cd5e2efec0e0b0180bb068f25d2e39","tier":null}],"members":[{"type":"individual","id":"da05e359859db73e","login":"benjamintoofer","name":"","avatar_url":"https://avatars-next.observableusercontent.com/e301ce5d3ad1a42b0de3f694a65c97ff71cd5e2efec0e0b0180bb068f25d2e39","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":1,"collection":null,"type":"notebook","body":{"titling":"auto","title":"ISO Base Media File Format","cells":[{"id":0,"value":"md`# ISO Base Media File Format`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":529,"value":"html`\n\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":523,"value":"$32","pinned":false,"hidden":false,"mode":"html","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":8,"value":"$33","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":51,"value":"$34","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":116,"value":"$35","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":235,"value":"md`\n## Movie Box: \\`moov\\`\n---\n### Description\n> \nThis box is placed at the top level. This is a mandatory box and their must be exactly one box.This box conatins other boxes that have the detailed metadata information about the presentation.\n\n\n### Class View\n> \\`\\`\\`javascript\nclass MOOV extends Box {\n // Contains other boxes\n}\n\\`\\`\\`\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":265,"value":"$36","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":124,"value":"md`\n## Track Box: \\`trak\\`\n---\n\n### Description\n> \nThis is a container box listed under the \\`moov\\` box (Movie Box) and it is mandatory. It represents a single track of a presentation (Video, Audio, Text). Each track is indepentdent from the other tracks and will carry its own information.

\nThere are two purposes for a track:\n>>1. Contain media data (in the \\`mdia\\` box)
\n 2. Contain packetization information for hint tracks\n\n\n**TODO FINISH DESCRIPTION**\n\n### Class View\n> \\`\\`\\`javascript\nclass TRAK extends Box {\n // Contains other boxes\n}\n\\`\\`\\`\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":471,"value":"$37","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":585,"value":"md`\n## Edit Box: \\`edts\\`\n---\n### Description\n> \nThis box is placed within the \\`trak\\` box (Track Box). It is not mandatory. It is a container for edit lists. If this box is absent, it is assumed that there is a 1-to-1 mapping of the video and audio timelines.\n\n### Class View\n> \\`\\`\\`javascript\nclass EDTS extends Box {\n // Contains other boxes\n}\n\\`\\`\\`\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":598,"value":"md`\n## Edit List Box: \\`elst\\`\n---\n### Description\n> \nThis box is placed within the \\`edts\\` box (Edit Box). It is not mandatory. \n\n### Class View\n> \\`\\`\\`javascript\nclass EDTS extends FullBox {\n // Contains other boxes\n}\n\\`\\`\\`\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":422,"value":"md`\n## Movie Extends Box: \\`mvex\\`\n---\n### Description\n> \nThis box is placed within the \\`moov\\` box (Movie Box). It notifies the reader that there might be \\`moof\\` boxes (Movie Fragment Boxes). Should be in the intialization header (I think?)\n\n\n### Class View\n> \\`\\`\\`javascript\nclass MVEX extends Box {\n // Contains other boxes\n}\n\\`\\`\\`\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":425,"value":"$38","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":121,"value":"md`\n## Media Data Box: \\`mdat\\`\n---\n### Description\n> \nThis box contains the media data. This data forms the frames fr plaback. There are other boxes that contain just metadata about how the data in this box is stored so that it makes it possible to understand how the data is orgranized and how to parse it.\n\n\n### Properties\n> **Data**: All the data for video and audio are contained in this box. It holds data for multiple frames.\n
\n\n### Class View\n> \\`\\`\\`javascript\nclass MDAT extends Box {\n // 8 unsigned bit []\n}\n\\`\\`\\`\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":126,"value":"md`\n## Movie Fragment Header Box: \\`mfhd\\`\n---\n### Description\n> \nThis box\n\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":119,"value":"md`\n## Movie Fragment Box: \\`moof\\`\n---\n### Description\n> \nThis box is top level. This box was added to ISO-BMFF standard for streaming. \\`moov\\` (MovieBox) contained the metadata for the whole movie but \\`moof\\` was added to allow the tracks (Audio, Video, and Text) to be broken up into framgents (That's why it is called Movie Fragment Box) and contain the metadata for each track. There MUST be one of these boxes per fragment.\n\n\n### Class View\n> \\`\\`\\`javascript\nclass MovieFragmentBox extends Box {\n // Contains other boxes\n}\n\\`\\`\\`\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":140,"value":"md`\n## Track Fragment Box: \\`traf\\`\n---\n### Description\n> \nThis box is contained within the \\`moof\\` box. There must be one of these boxes per \\'moof\\'. This box is the container to other metadata boxes, specifically track metadata. Its contains a single \\`tfhd\\` box and can contain multiple \\`trun\\` boxes.\n\n\n### Class View\n> \\`\\`\\`javascript\nclass TrackFragmentBox extends Box {\n // Contains other boxes\n}\n\\`\\`\\`\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":321,"value":"$39","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":158,"value":"md`\n## Track Fragment Run Boxes: \\`trun\\`\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":128,"value":"md`\n## Protection System Specific Header: \\`pssh\\`\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":130,"value":"md`\n## Segment Index Box: \\`sidx\\`\n---\n### Description\n> \nThis box is for media streaming. It is an alternative way to requesting segments. It provides the metadata to \n\n\n### Class View\n> \\`\\`\\`javascript\nclass SIDX extends FullBox {\n referenceID; // 32 unsigned bit\n}\n\\`\\`\\`\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":134,"value":"md`\n## Sample Encryption Box: \\`senc\\`\n\n> This is an extention to the \\`Box\\`. Fullboxes contain an additional **size** and **type**\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W3a","stdlib":"1"}} 31:"$30:body"