ELF Symbol Upload

sentry-cli can upload ELF symbols generated on various Linux distributions to Sentry to allow symbolication of Linux and Android app crashes. ELF stands for Executable and Linkable Format, the file format used for binaries on Linux.

Unlike other platforms, there is no standardized container for debug symbols. They are part of the binary (executable or library) and stripped when generating release builds due to their size. However, there is a way to retain them in a separate file (either in a different location or with .debug extension):

# There is an executable called "binary" in the CWD
objcopy --only-keep-debug binary binary.debug
objcopy --strip-debug --strip-unneeded binary
objcopy --add-gnu-debuglink=binary.debug binary

Shared libraries installed via package managers usually provide their debugging information in separate *-dev packages and put it in locations like /usr/local/debug/.... To receive symbolicated stack traces from those libraries, make sure to also upload their symbols in addition to your app’s symbols.

Build Identifiers

Sentry uses the GNU build identifier to associate uploaded debug information files with crashes. All recent compilers and linkers support the emission of build IDs, but sometimes they might require additional configuration. gcc does this by default, for clang use one of the following flags:

  • --build-id=uuid for a fast but non-reproducible random identifier.
  • --build-id=sha1 for a slower but reproducible identifier generated by hashing the first page of the code section.

Note that the identifier needs to be present and identical in the binary as well as stripped debug information files. If the ID is missing for some reason, upload the files before stripping so that sentry-cli can compute a stable identifier from the unstripped file. This can be verified with our difutil command:

 $ sentry-cli difutil check path/to/file

Debug Info File Check
  Type: elf library
  Contained debug identifiers:
    > 924e148f-3bb7-06a0-74c1-36f42f08b40e (x86_64)
  ...

Debug Info Compression

ELF supports the compression of debug information which can significantly reduce the time required to upload debug information files to Sentry and thus improve build times. gcc (version 5 or newer) and clang (version 5 or newer) support this by passing the -gz flag to both the compiler and linker.

This can be verified by checking for the C flag in readelf, corresponding to SHF_COMPRESSED:

$ readelf -S path/to/file
  ...
  [21] .debug_info       PROGBITS         0000000000000000  00000370
       000000000000e133  0000000000000000   C       0     0     1

Basic Upload

Use upload-dif to upload ELF symbols and specify the elf type. The command will recurively scan the provided folders or ZIP archives. If you stripped debug information into separate files, this uploads both the stripped executable and the debug information file to achieve highest stack trace quality. This is particularly helpful for release builds with frame pointer omission (FPO).

Example:

$ sentry-cli upload-dif -t elf .

> Found 2 debug information files
> Prepared debug information files for upload
> Uploaded 2 missing debug information files
> File processing complete:

     OK c0bcc3f1-9827-fe65-3058-404b2831d9e6 (binary; x86_64 executable)
     OK c0bcc3f1-9827-fe65-3058-404b2831d9e6 (binary.debug; x86_64 debug companion)

Upload Options

There are a few options you can supply for the upload process

--no-unwind

Do not scan for stack unwinding information. Specify this flag for builds with disabled FPO, or when stackwalking occurs on the device. This usually excludes executables and libraries. They might still be uploaded, if they contain debug information.

--no-debug

Do not scan for debug information. This will usually exclude debug companion files. They might still be uploaded, if they contain stack unwinding information.

--no-zips

By default, sentry-cli will open and search ZIP archives for files. Use this switch to disable if your search paths contain large ZIP archives without debug information files to speed up the search.

--no-reprocessing

This parameter prevents Sentry from triggering reprocessing right away. It can be useful under rare circumstances where you want to upload files in multiple batches and you want to ensure that Sentry does not start reprocessing before some optional dsyms are uploaded. Note though that someone can still in the meantime trigger reprocessing from the UI.