Integrate new cross-platform symbolicator #1800
Conversation
- add a new migrator service - setup a common migration infrastructure for migrating objects - add a journal component to keep track of migrations Signed-off-by: detj <[email protected]>
Signed-off-by: detj <[email protected]>
- modify builds api to process ios build mapping files - support both ios and android platform Signed-off-by: detj <[email protected]>
- add tests for dsym debug information file processing Signed-off-by: detj <[email protected]>
Signed-off-by: detj <[email protected]>
- integrate symbolicator service - figure out iOS symbolication - update sdk api docs fixes #1320 Signed-off-by: detj <[email protected]>
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| return | ||
| } | ||
|
|
||
| fmt.Println("sources:", string(sources)) |
Check failure
Code scanning / CodeQL
Clear-text logging of sensitive information High
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix AI 18 days ago
The best way to fix the problem is to avoid logging sensitive information such as SecretKey and AccessKey. Instead, we can log a message indicating that the sources were processed without including the sensitive details. This can be achieved by removing the logging of the sources variable and replacing it with a more generic log message.
To implement the changes, we need to modify the makeRequest method in the backend/api/symbolicator/symbolicator.go file. Specifically, we will remove the line that logs the sources variable and replace it with a more generic log message.
| @@ -59,3 +59,3 @@ | ||
|
|
||
| fmt.Println("sources:", string(sources)) | ||
| fmt.Println("sources processed successfully") | ||
| } |
Signed-off-by: detj <[email protected]>
Signed-off-by: detj <[email protected]>
Signed-off-by: detj <[email protected]>
- add support for uploading multiple mapping files for iOS dSYM files Signed-off-by: detj <[email protected]>
| code = http.StatusBadRequest | ||
| maxSize := int64(server.Server.Config.MappingFileMaxSize) |
Check failure
Code scanning / CodeQL
Incorrect conversion between integer types High
Signed-off-by: detj <[email protected]>
- update sessionator record command to support uploading multiple mapping files - fix an issue where mapping type was not being read properly - fix an issue where optional build size was not being written properly Signed-off-by: detj <[email protected]>
Signed-off-by: detj <[email protected]>
| } | ||
|
|
||
| mappingFilePath := filepath.Join(outputDir, appUniqueID, versionName, versionCode, filename) | ||
| if err := os.MkdirAll(filepath.Dir(mappingFilePath), 0755); err != nil { |
Check failure
Code scanning / CodeQL
Uncontrolled data used in path expression High
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix AI 5 days ago
To fix the problem, we need to validate the appUniqueID, versionName, and versionCode values to ensure they do not contain any path traversal characters or sequences. This can be done by checking for the presence of path separators ("/" or "\") and ".." sequences, and rejecting the input if any are found. Additionally, we should ensure that the resolved path is within a specific safe directory.
| @@ -307,2 +307,22 @@ | ||
|
|
||
| // Validate inputs to prevent path traversal | ||
| if strings.Contains(appUniqueID, "/") || strings.Contains(appUniqueID, "\\") || strings.Contains(appUniqueID, "..") { | ||
| c.JSON(http.StatusBadRequest, gin.H{ | ||
| "error": "Invalid app_unique_id", | ||
| }) | ||
| return | ||
| } | ||
| if strings.Contains(versionName, "/") || strings.Contains(versionName, "\\") || strings.Contains(versionName, "..") { | ||
| c.JSON(http.StatusBadRequest, gin.H{ | ||
| "error": "Invalid version_name", | ||
| }) | ||
| return | ||
| } | ||
| if strings.Contains(versionCode, "/") || strings.Contains(versionCode, "\\") || strings.Contains(versionCode, "..") { | ||
| c.JSON(http.StatusBadRequest, gin.H{ | ||
| "error": "Invalid version_code", | ||
| }) | ||
| return | ||
| } | ||
|
|
||
| if appUniqueID == "" { | ||
| @@ -375,3 +395,10 @@ | ||
| mappingFilePath := filepath.Join(outputDir, appUniqueID, versionName, versionCode, filename) | ||
| if err := os.MkdirAll(filepath.Dir(mappingFilePath), 0755); err != nil { | ||
| absMappingFilePath, err := filepath.Abs(mappingFilePath) | ||
| if err != nil || !strings.HasPrefix(absMappingFilePath, outputDir) { | ||
| c.JSON(http.StatusBadRequest, gin.H{ | ||
| "error": "Invalid file path", | ||
| }) | ||
| return | ||
| } | ||
| if err := os.MkdirAll(filepath.Dir(absMappingFilePath), 0755); err != nil { | ||
| c.JSON(http.StatusInternalServerError, gin.H{ |
| defer enc.Close() | ||
| _, err = io.Copy(enc, file) | ||
|
|
||
| out, err := os.Create(mappingFilePath) |
Check failure
Code scanning / CodeQL
Uncontrolled data used in path expression High
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix AI 5 days ago
To fix the problem, we need to validate the user input before using it to construct the file path. Specifically, we should ensure that the appUniqueID, versionName, and versionCode values do not contain any path separators or ".." sequences. This can be done by checking for the presence of these characters and rejecting the input if any are found.
| @@ -303,4 +303,22 @@ | ||
| appUniqueID := c.Request.FormValue("app_unique_id") | ||
| if strings.Contains(appUniqueID, "/") || strings.Contains(appUniqueID, "\\") || strings.Contains(appUniqueID, "..") { | ||
| c.JSON(http.StatusBadRequest, gin.H{ | ||
| "error": "Invalid app_unique_id", | ||
| }) | ||
| return | ||
| } | ||
| versionName := c.Request.FormValue("version_name") | ||
| if strings.Contains(versionName, "/") || strings.Contains(versionName, "\\") || strings.Contains(versionName, "..") { | ||
| c.JSON(http.StatusBadRequest, gin.H{ | ||
| "error": "Invalid version_name", | ||
| }) | ||
| return | ||
| } | ||
| versionCode := c.Request.FormValue("version_code") | ||
| if strings.Contains(versionCode, "/") || strings.Contains(versionCode, "\\") || strings.Contains(versionCode, "..") { | ||
| c.JSON(http.StatusBadRequest, gin.H{ | ||
| "error": "Invalid version_code", | ||
| }) | ||
| return | ||
| } | ||
| mappingType := c.Request.FormValue("mapping_type") |
| buildFilePath := filepath.Join(outputDir, appUniqueID, versionName, "build.toml") | ||
| out, err = os.Create(buildFilePath) | ||
| buildFilePath := filepath.Join(outputDir, appUniqueID, versionName, versionCode, "build.toml") | ||
| if err := os.MkdirAll(filepath.Dir(buildFilePath), 0755); err != nil { |
Check failure
Code scanning / CodeQL
Uncontrolled data used in path expression High
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix AI 5 days ago
To fix the problem, we need to ensure that the user-provided appUniqueID is validated before being used to construct file paths. We can achieve this by checking that the appUniqueID does not contain any path separators or parent directory references. Additionally, we should ensure that the resolved path is within a specific safe directory.
- Validate the
appUniqueIDto ensure it does not contain any path separators or parent directory references. - Ensure that the resolved path is within a specific safe directory.
| @@ -303,2 +303,8 @@ | ||
| appUniqueID := c.Request.FormValue("app_unique_id") | ||
| if strings.Contains(appUniqueID, "/") || strings.Contains(appUniqueID, "\\") || strings.Contains(appUniqueID, "..") { | ||
| c.JSON(http.StatusBadRequest, gin.H{ | ||
| "error": "Invalid app_unique_id", | ||
| }) | ||
| return | ||
| } | ||
| versionName := c.Request.FormValue("version_name") | ||
| @@ -412,3 +418,10 @@ | ||
| buildFilePath := filepath.Join(outputDir, appUniqueID, versionName, versionCode, "build.toml") | ||
| if err := os.MkdirAll(filepath.Dir(buildFilePath), 0755); err != nil { | ||
| absBuildFilePath, err := filepath.Abs(buildFilePath) | ||
| if err != nil || !strings.HasPrefix(absBuildFilePath, outputDir) { | ||
| c.JSON(http.StatusBadRequest, gin.H{ | ||
| "error": "Invalid build file path", | ||
| }) | ||
| return | ||
| } | ||
| if err := os.MkdirAll(filepath.Dir(absBuildFilePath), 0755); err != nil { | ||
| c.JSON(http.StatusInternalServerError, gin.H{ |
| }) | ||
| return | ||
| } | ||
| out, err := os.Create(buildFilePath) |
Check failure
Code scanning / CodeQL
Uncontrolled data used in path expression High
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix AI 5 days ago
To fix the problem, we need to validate the user input before using it to construct a file path. Specifically, we should ensure that the appUniqueID does not contain any path separators or parent directory references. This can be done by checking for the presence of "/" or "\" characters and ".." sequences in the appUniqueID value. If any of these are found, we should reject the input and return an error response.
| @@ -313,2 +313,9 @@ | ||
| } | ||
| // Validate appUniqueID to ensure it does not contain any path separators or parent directory references | ||
| if strings.Contains(appUniqueID, "/") || strings.Contains(appUniqueID, "\\") || strings.Contains(appUniqueID, "..") { | ||
| c.JSON(http.StatusBadRequest, gin.H{ | ||
| "error": "Invalid app_unique_id", | ||
| }) | ||
| return | ||
| } | ||
| if versionName == "" { |
- add environment variable for symbolicator origin in server config Signed-off-by: detj <[email protected]>
Signed-off-by: detj <[email protected]>
Signed-off-by: detj <[email protected]>
Signed-off-by: detj <[email protected]>
Summary
This PR introduces a new cross-platform symbolicator service and replaces the old symbolication pipeline with a new one.
Tasks
binary_imagesfor iOS exceptionsexceptionevent schema for iOSbinary_imagessection in event schema for iOSImplement a new symbolication batching pipelinebinary_imagesfor iOS exceptions to ClickHouseSee also