You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Create a Snapshot model that includes the following fields:
start_at: A datetime field indicating when the snapshot period begins.
end_at: A datetime field indicating when the snapshot period ends.
status: One of "pending", "processing", "completed", or "error".
new_chapters: A many-to-many relationship with the OWASP Chapter model.
new_projects: A many-to-many relationship with the OWASP Project model.
new_issues: A many-to-many relationship with the GitHub Issue model.
new_releases: A many-to-many relationship with the GitHub Release model.
new_users: A many-to-many relationship with the GitHub User model.
This model will serve as a record for each snapshot period, capturing the new entities within that timeframe.
Management Command Design
The management command will perform the following steps:
Fetch Unprocessed Snapshots:
Query the database for all Snapshot records where the status is "pending". These are the snapshots that have not yet been populated with new data.
Update Status to Processing:
For each pending snapshot, update the status to "processing" before starting the data population. This ensures that multiple runs of the command do not process the same snapshot concurrently.
Populate Snapshot Data:
Using the start_at and end_at values of the snapshot, query each of the associated models (Chapters, Projects, GitHub Issues, GitHub Releases, and GitHub Users) for objects created or updated within that time window.
Populate the respective many-to-many relationships of the snapshot with the newly retrieved data.
Update Snapshot Status:
Once the snapshot has been successfully populated with data, update the status to "completed".
If an error occurs during processing, mark the snapshot with a status of "error" for further review.
Logging and Reporting:
The command should log its progress, including which snapshots were processed successfully and which encountered issues. This information will be valuable for debugging and for ensuring that all unprocessed snapshots are handled appropriately.
The text was updated successfully, but these errors were encountered:
Describe the solution you'd like
In
owasp
application:Snapshot Model
Create a Snapshot model that includes the following fields:
This model will serve as a record for each snapshot period, capturing the new entities within that timeframe.
Management Command Design
The management command will perform the following steps:
Fetch Unprocessed Snapshots:
Query the database for all Snapshot records where the
status
is "pending". These are the snapshots that have not yet been populated with new data.Update Status to Processing:
For each pending snapshot, update the status to "processing" before starting the data population. This ensures that multiple runs of the command do not process the same snapshot concurrently.
Populate Snapshot Data:
start_at
andend_at
values of the snapshot, query each of the associated models (Chapters, Projects, GitHub Issues, GitHub Releases, and GitHub Users) for objects created or updated within that time window.Update Snapshot Status:
Logging and Reporting:
The command should log its progress, including which snapshots were processed successfully and which encountered issues. This information will be valuable for debugging and for ensuring that all unprocessed snapshots are handled appropriately.
The text was updated successfully, but these errors were encountered: