fix: Set DatabricksConnectionConfig to only use a shared_connection with U2M OAuth#5859
Conversation
…hen U2M OAuth authentication is used. Signed-off-by: davem-bis <68955845+davem-bis@users.noreply.github.com>
…databricks-access-token-connections
|
@davem-bis Thanks for the PR! Could you look into the merge conflict and failed ci tests? |
…-shared-connection-for-databricks-access-token-connections # Conflicts: # sqlmesh/core/config/connection.py
…nection-for-databricks-access-token-connections' into feature/DRM/revert-to-shared-connection-for-databricks-access-token-connections
Signed-off-by: davem-bis <68955845+davem-bis@users.noreply.github.com>
|
@davem-bis Could you look into failed ci tests? |
|
Hi @StuffbyYuki - will do, I'm still working on the branch, didn't mean to push the changes yet! |
Signed-off-by: davem-bis <68955845+davem-bis@users.noreply.github.com>
641f480 to
542ced4
Compare
…-shared-connection-for-databricks-access-token-connections
Signed-off-by: davem-bis <68955845+davem-bis@users.noreply.github.com>
|
Hi @StuffbyYuki - I've should have resolved the test failure from the CI. I haven't been able to get all of the tests to pass locally, but I have at least managed to get the number of failed tests to match that on main! FTAOD, the tests I have failing locally all look to be passing in the CI. Once I've got a dev container that can run all of the tests locally I'll submit another PR with that in. |
|
I reviewed the pr. Not a blocker, but maybe align U2M detection with _static_connection_kwargs using not oauth_client_secret instead of oauth_client_id is None? Let me know what you think |
Description
The DatabricksConnectionConfig was set to use a shared connection pool for all authentication types to resolve a race condition with U2M OAuth, however this has the side effect that SQL execution is sequential regardless of the
concurrent_tasksconfig setting.This change updates:
shared_connectionvariable to be a computed field for allConnectionConfigclasses.shared_connectionvariable to returnTrueonly for U2M OAuth authentication methods for theDatabricksConnectionConfigclass.Partially resolves #5858.
Test Plan
Checklist
make styleand fixed any issuesmake fast-test)git commit -s) per the DCO