Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Incremental Metadata Extraction is dispalyed for sources that are not supported #17397

Open
sushi30 opened this issue Aug 12, 2024 · 2 comments · May be fixed by #19468
Open

Incremental Metadata Extraction is dispalyed for sources that are not supported #17397

sushi30 opened this issue Aug 12, 2024 · 2 comments · May be fixed by #19468
Assignees
@shrushti2000
Labels
P0 Highest priority UI UI specific issues
Milestone
Feb 25 - Sprint 1

Comments

@sushi30
Copy link
Contributor

sushi30 commented Aug 12, 2024 •
edited
Loading

Affected module
Ingestion, UI

Describe the bug
Incremental metadata extraction is supported only for some the database sources. Namely, those that store DDL and DML queries in some log. This configuration should not be available for databases that dont support it (like MySQL, Postgres, MSSQL).

When enabled for unsupported sources this configuration does not change the behavior of the ingestion process.

To Reproduce

image

Expected behavior
A clear and concise description of what you expected to happen.

Version:

  • OS: [e.g. iOS]
  • Python version:
  • OpenMetadata version: [e.g. 0.8]
  • OpenMetadata Ingestion package version: [e.g. openmetadata-ingestion[docker]==XYZ]

Additional context
Add any other context about the problem here.
@sushi30 sushi30 changed the title Incremental Metadata Extracaction is dispalyed for sources that are not supported Incremental Metadata Extraction is dispalyed for sources that are not supported Aug 12, 2024
@pmbrull pmbrull added the UI UI specific issues label Aug 12, 2024
@pmbrull
Copy link
Collaborator

pmbrull commented Aug 12, 2024 •
edited
Loading

as part of this, we should think how to generalize/improve ingestion config handling on the UI:

  1. how to display/hide not supported options
  2. how to better group configurations: Filters, Metadata, Security, Advanced Options
  3. Improve Filters to be added instead of ocupying a big chunk of the page
  4. We can try to handle better defaults in filtering. E.g., excluding INFORMATION_SCHEMA by default

Basically the configs now seem super complicated, we should simplify it while keeping it flexible for more advanced users

@harshach harshach moved this to Integration in Release 1.7.0 Dec 14, 2024
@harshach harshach added the P0 Highest priority label Dec 14, 2024
@harshach harshach added this to the Feb 25 - Sprint 1 milestone Jan 21, 2025
@harshach
Copy link
Collaborator

@chirag-madlani re-assing

@shrushti2000 shrushti2000 linked a pull request Jan 21, 2025 that will close this issue
Open
9 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@shrushti2000 shrushti2000

Labels
P0 Highest priority UI UI specific issues
Projects
Release 1.7.0
Status: Integration
Milestone
Feb 25 - Sprint 1
5 participants
@harshach @sushi30 @pmbrull @Sachin-chaurasiya @shrushti2000