|
Lines 2-13
Link Here
|
| 2 |
GLEP: 78 |
2 |
GLEP: 78 |
| 3 |
Title: Gentoo binary package container format |
3 |
Title: Gentoo binary package container format |
| 4 |
Author: Michał Górny <mgorny@gentoo.org> |
4 |
Author: Michał Górny <mgorny@gentoo.org> |
|
|
5 |
Sheng Yu <syu.os@protonmail.com> |
| 5 |
Type: Standards Track |
6 |
Type: Standards Track |
| 6 |
Status: Draft |
7 |
Status: Draft |
| 7 |
Version: 1 |
8 |
Version: 1 |
| 8 |
Created: 2018-11-15 |
9 |
Created: 2018-11-15 |
| 9 |
Last-Modified: 2019-07-29 |
10 |
Last-Modified: 2021-10-10 |
| 10 |
Post-History: 2018-11-17, 2019-07-08 |
11 |
Post-History: 2018-11-17, 2019-07-08, 2021-09-13, 2021-09-22, 2022-05-28 |
| 11 |
Content-Type: text/x-rst |
12 |
Content-Type: text/x-rst |
| 12 |
--- |
13 |
--- |
| 13 |
|
14 |
|
|
Lines 154-163
The following obligatory goals have been set for a replacement format:
Link Here
|
| 154 |
enough to let user inspect and manipulate it without special tooling |
155 |
enough to let user inspect and manipulate it without special tooling |
| 155 |
or detailed knowledge. |
156 |
or detailed knowledge. |
| 156 |
|
157 |
|
| 157 |
3. **The file format must provide support for OpenPGP signatures.** |
158 |
3. **The file format must be able to detect its own data corruption.** |
|
|
159 |
In particular, it needs to contain the checksum of its own data for |
| 160 |
package manager to be able to verify its integrity without relying |
| 161 |
on additional files. |
| 162 |
|
| 163 |
4. **The file format must provide support for OpenPGP signatures.** |
| 158 |
Preferably, it should use standard OpenPGP message formats. |
164 |
Preferably, it should use standard OpenPGP message formats. |
| 159 |
|
165 |
|
| 160 |
4. **The file format must allow for efficient metadata updates.** |
166 |
5. **The file format must allow for efficient metadata updates.** |
| 161 |
In particular, it should be possible to update the metadata without |
167 |
In particular, it should be possible to update the metadata without |
| 162 |
having to recompress package files. |
168 |
having to recompress package files. |
| 163 |
|
169 |
|
|
Lines 186-220
The container format
Link Here
|
| 186 |
The gpkg package container is an uncompressed .tar achive whose filename |
192 |
The gpkg package container is an uncompressed .tar achive whose filename |
| 187 |
should use ``.gpkg.tar`` suffix. |
193 |
should use ``.gpkg.tar`` suffix. |
| 188 |
|
194 |
|
| 189 |
The archive contains a number of files, stored in a single directory |
195 |
The archive contains a number of files. All package-related files |
| 190 |
whose name should match the basename of the package file. However, |
196 |
should be stored in a single directory whose name matches the basename |
| 191 |
the implementation must be able to process an archive where |
197 |
of the package file. However, the implementation must be able to |
| 192 |
the directory name is mismatched. There should be no explicit archive |
198 |
process an archive where the directory name is mismatched. There should |
| 193 |
member entry for the directory. |
199 |
be no explicit archive member entry for the directory. |
| 194 |
|
200 |
|
| 195 |
The package directory contains the following members, in order: |
201 |
The package directory contains the following members, in order: |
| 196 |
|
202 |
|
| 197 |
1. The package format identifier file ``gpkg-1`` (required). |
203 |
1. The package format identifier file ``gpkg-1`` (required). |
| 198 |
|
204 |
|
| 199 |
2. A signature for the metadata archive: ``metadata.tar${comp}.sig`` |
205 |
2. The metadata archive ``metadata.tar${comp}``, optionally compressed |
|
|
206 |
(required). |
| 207 |
|
| 208 |
3. A signature for the metadata archive: ``metadata.tar${comp}.sig`` |
| 200 |
(optional). |
209 |
(optional). |
| 201 |
|
210 |
|
| 202 |
3. The metadata archive ``metadata.tar${comp}``, optionally compressed |
211 |
4. The filesystem image archive ``image.tar${comp}``, optionally |
| 203 |
(required). |
212 |
compressed (required). |
| 204 |
|
213 |
|
| 205 |
4. A signature for the filesystem image archive: |
214 |
5. A signature for the filesystem image archive: |
| 206 |
``image.tar${comp}.sig`` (optional). |
215 |
``image.tar${comp}.sig`` (optional). |
| 207 |
|
216 |
|
| 208 |
5. The filesystem image archive ``image.tar${comp}``, optionally |
217 |
6. The package Manifest data file ``Manifest``, optionally clear-text |
| 209 |
compressed (required). |
218 |
signed (required) |
| 210 |
|
219 |
|
| 211 |
It is recommended that relative order of the archive members is |
220 |
It is recommended that relative order of the archive members is |
| 212 |
preserved. However, implementations must support archives with members |
221 |
preserved. However, implementations must support archives with members |
| 213 |
out of order. |
222 |
out of order. |
| 214 |
|
223 |
|
| 215 |
The container may be extended with additional members in the future. |
224 |
The container may be extended with additional members in the future. |
| 216 |
The implementations should ignore unrecognized members and preserve |
225 |
If the Manifest is present, all files contained in the archive must |
| 217 |
them across package updates. |
226 |
be listed in it and verify successfully. The package manager should |
|
|
227 |
ignore unknown files but preserve them across package updates. |
| 218 |
|
228 |
|
| 219 |
|
229 |
|
| 220 |
Permitted .tar format features |
230 |
Permitted .tar format features |
|
Lines 301-310
suffixed using the standard suffix for the particular compressed file
Link Here
|
| 301 |
type (e.g. ``.bz2`` for bzip2 format). |
311 |
type (e.g. ``.bz2`` for bzip2 format). |
| 302 |
|
312 |
|
| 303 |
|
313 |
|
|
|
314 |
The package Manifest file |
| 315 |
------------------------- |
| 316 |
|
| 317 |
The Manifest file must include digests of all files in the binary |
| 318 |
package container, except for itself. The purpose of this file is |
| 319 |
to provide the package manager with an ability to detect corruption |
| 320 |
or alteration of the binary package before attempting to read the |
| 321 |
inner archive contents. This file also provides protection against |
| 322 |
signature reuse/replacement attacks if the OpenPGP signatures are used. |
| 323 |
|
| 324 |
The implementation follows the Manifest specifications in GLEP 74 |
| 325 |
[#GLEP74]_ and uses the DATA tag for files within the container. |
| 326 |
|
| 327 |
The implementation should be able to detect checksum mismatches, |
| 328 |
as well as missing, duplicate, or extraneous files within the |
| 329 |
container. In the case of verification failure, no subsequent |
| 330 |
operations on the archive should be performed. |
| 331 |
|
| 332 |
|
| 304 |
OpenPGP member signatures |
333 |
OpenPGP member signatures |
| 305 |
------------------------- |
334 |
------------------------- |
| 306 |
|
335 |
|
| 307 |
The archive members support optional OpenPGP signatures. |
336 |
The archive members and Manifest support optional OpenPGP signatures. |
| 308 |
The implementations must allow the user to specify whether OpenPGP |
337 |
The implementations must allow the user to specify whether OpenPGP |
| 309 |
signatures are to be expected in remotely fetched packages. |
338 |
signatures are to be expected in remotely fetched packages. |
| 310 |
|
339 |
|
|
Lines 490-495
Debian has a similar guideline for the inner tar of their package
Link Here
|
| 490 |
format [#DEB-FORMAT]_. |
519 |
format [#DEB-FORMAT]_. |
| 491 |
|
520 |
|
| 492 |
|
521 |
|
|
|
522 |
.tar security issues |
| 523 |
-------------------- |
| 524 |
|
| 525 |
Some of the original features of .tar are obsolete with the modern |
| 526 |
usage. |
| 527 |
|
| 528 |
Firstly, .tar permits duplicate files to exist [#TARDUP]_. The |
| 529 |
later duplicate files overwrite the previously extracted files when |
| 530 |
extracting all files in order. This is useful for incremental |
| 531 |
backups. However, a general-purpose archiving tools may choose |
| 532 |
arbitrary files matching a path name, leading to checksum or |
| 533 |
signature bypass. To prevent this, duplicate files are forbidden |
| 534 |
from existing. |
| 535 |
|
| 536 |
Secondly, .tar lacks integrity checks, except for the header |
| 537 |
self-check. Data corruption can usually be detected through |
| 538 |
integrity checks in the additional compression layer. However, |
| 539 |
this does not provide a way of verifying the integrity of the |
| 540 |
compressed data in advance. For this reason, an additional |
| 541 |
Manifest file is included that provides checksums for other |
| 542 |
files in the archive. A corrupted Manifest invalidates the whole |
| 543 |
package. |
| 544 |
|
| 545 |
Thirdly, many .tar implementations have various security problems, |
| 546 |
including the Python tarfile module [#ISSUE21109]_. They provide |
| 547 |
multiple attack vectors, e.g. permitting overwriting files outside the |
| 548 |
destination directory using special filenames, symlinks, hard links or |
| 549 |
device files. For this purpose, only regular files are permitted inside |
| 550 |
the container. It is recommended to process the container data in place |
| 551 |
rather than extracting it. |
| 552 |
|
| 553 |
|
| 493 |
Member ordering |
554 |
Member ordering |
| 494 |
--------------- |
555 |
--------------- |
| 495 |
|
556 |
|
|
Lines 511-516
them. Covering the compressed archives helps to prevent zipbomb
Link Here
|
| 511 |
attacks. Covering the individual members rather than the whole package |
572 |
attacks. Covering the individual members rather than the whole package |
| 512 |
provides for verification of partially fetched binary packages. |
573 |
provides for verification of partially fetched binary packages. |
| 513 |
|
574 |
|
|
|
575 |
However, signing individual files does not guarantee that all members |
| 576 |
are originating from the same binary package. This opens up the |
| 577 |
possibility of a replacement/reuse attack, e.g. combining the signed |
| 578 |
metadata from foo-1.1 with signed image from foo-1.0. The new binary |
| 579 |
package passes the signature check. To prevent this type of attack, |
| 580 |
we need the additional Menifest file and its signature to verify the |
| 581 |
authenticity of the complete binary package. |
| 582 |
|
| 514 |
|
583 |
|
| 515 |
Format versioning |
584 |
Format versioning |
| 516 |
----------------- |
585 |
----------------- |
|
Lines 564-573
References
Link Here
|
| 564 |
.. [#TAR-PORTABILITY] Michał Górny, Portability of tar features |
633 |
.. [#TAR-PORTABILITY] Michał Górny, Portability of tar features |
| 565 |
(https://dev.gentoo.org/~mgorny/articles/portability-of-tar-features.html) |
634 |
(https://dev.gentoo.org/~mgorny/articles/portability-of-tar-features.html) |
| 566 |
|
635 |
|
|
|
636 |
.. [#GLEP74] GLEP 74: Full-tree verification using Manifest files |
| 637 |
(https://www.gentoo.org/glep/glep-0074.html) |
| 638 |
|
| 567 |
.. [#XPAK2GPKG] xpak2gpkg: Proof-of-concept converter from tbz2/xpak |
639 |
.. [#XPAK2GPKG] xpak2gpkg: Proof-of-concept converter from tbz2/xpak |
| 568 |
to gpkg binpkg format |
640 |
to gpkg binpkg format |
| 569 |
(https://github.com/mgorny/xpak2gpkg) |
641 |
(https://github.com/mgorny/xpak2gpkg) |
| 570 |
|
642 |
|
|
|
643 |
.. [#TARDUP] tar: Multiple Members with the Same Name |
| 644 |
(https://www.gnu.org/software/tar/manual/html_node/multiple.html) |
| 645 |
|
| 646 |
.. [#ISSUE21109] Python tarfile: Traversal attack vulnerability |
| 647 |
(https://bugs.python.org/issue21109) |
| 648 |
|
| 571 |
|
649 |
|
| 572 |
Copyright |
650 |
Copyright |
| 573 |
========= |
651 |
========= |
| 574 |
- |
|
|
