Fix the way we write configure-kube-proxy.sh

[gabriel-samfira]

On flatcar, files in /run are recreated after the initrd pivots to the full system and executes init. This change adds a unit file that writes the file post-boot. Files written in /run during ignition run will be clobbered.

Fixes #480

[mnaser]

@gabriel-samfira do you happen to know why the flatcar CI fails? :(

[gabriel-samfira]

@mnaser looking

[gabriel-samfira]

The logs don't give much of a hint as to what the problem might be. I will try to run the tests on a local OpenStack deployment, connect to the VMs and debug.

Might take a few days. Need to finish some documentation.

[gabriel-samfira]

Hi @mnaser

I tried to run the integration tests using the

./hack/stack.sh
export KUBE_TAG=v1.27.4
./hack/run-integration-tests.sh

But I seem to be running into errors like:

Feb 11 11:44:09 test-bm2 magnum-conductor[2282404]: ERROR magnum.conductor.handlers.common.trust_manager keystoneauth1.exceptions.http.BadRequest: Invalid input for field/attribute trust. Value: {'roles': [{'id': '08cfd96596a445c493bca5af45593ea0', 'name': 'load-balancer_member', 'domain_id': None, 'description': None, 'options': {}, 'links': {'self': 'http://194.104.235.18/identity/v3/roles/08cfd96596a445c493bca5af45593ea0'}}, {'id': 'e463dc2438804143b2f2f44881beb0b5', 'name': 'reader', 'domain_id': None, 'description': None, 'options': {'immutable': True}, 'links': {'self': 'http://194.104.235.18/identity/v3/roles/e463dc2438804143b2f2f44881beb0b5'}}, {'id': '751a70d5b6ba4dd492ef0d2ee5aeb99e', 'name': 'member', 'domain_id': None, 'description': None, 'options': {'immutable': True}, 'links': {'self': 'http://194.104.235.18/identity/v3/roles/751a70d5b6ba4dd492ef0d2ee5aeb99e'}}, {'id': '12ba90b05b224b569812dbf10a2b6f06', 'name': 'anotherrole', 'domain_id': None, 'description': None, 'options': {}, 'links': {'self': 'http://194.104.235.18/identity/v3/roles/12ba90b05b224b569812dbf10a2b6f06'}}], 'delegation_depth': 0, 'id': '2ca7c13e7d7b4f568bd8d81cc096b136', 'trustor_user_id': '8285cc483d2c4899a3aa8e5efa331d88', 'trustee_user_id': '81b0e24a4d82446eb679433b2b004641', 'project_id': '04a0323b42a54819a0ecf51d8a3d8fc2', 'impersonation': True, 'expires_at': None, 'remaining_uses': None, 'deleted_at': None, 'redelegated_trust_id': None, 'redelegation_count': 0, 'roles_links': {'self': 'http://194.104.235.18/identity/v3/2ca7c13e7d7b4f568bd8d81cc096b136/roles', 'next': None, 'previous': None}, 'links': {'self': 'http://194.104.235.18/identity/v3/OS-TRUST/trusts/2ca7c13e7d7b4f568bd8d81cc096b136'}}. Additional properties are not allowed ('delegation_depth' was unexpected) (HTTP 400) (Request-ID: req-760632b6-16f8-4faa-ba32-f0e844f81e3f)
Feb 11 11:44:09 test-bm2 magnum-conductor[2282404]: ERROR magnum.conductor.handlers.common.trust_manager 
Feb 11 11:44:09 test-bm2 magnum-conductor[2282404]: ERROR magnum.conductor.handlers.common.trust_manager During handling of the above exception, another exception occurred:
Feb 11 11:44:09 test-bm2 magnum-conductor[2282404]: ERROR magnum.conductor.handlers.common.trust_manager 
Feb 11 11:44:09 test-bm2 magnum-conductor[2282404]: ERROR magnum.conductor.handlers.common.trust_manager Traceback (most recent call last):
Feb 11 11:44:09 test-bm2 magnum-conductor[2282404]: ERROR magnum.conductor.handlers.common.trust_manager   File "/opt/stack/magnum/magnum/conductor/handlers/common/trust_manager.py", line 34, in create_trustee_and_trust
Feb 11 11:44:09 test-bm2 magnum-conductor[2282404]: ERROR magnum.conductor.handlers.common.trust_manager     trust = osc.keystone().create_trust(
Feb 11 11:44:09 test-bm2 magnum-conductor[2282404]: ERROR magnum.conductor.handlers.common.trust_manager   File "/opt/stack/magnum/magnum/common/keystone.py", line 218, in create_trust
Feb 11 11:44:09 test-bm2 magnum-conductor[2282404]: ERROR magnum.conductor.handlers.common.trust_manager     raise exception.TrustCreateFailed(
Feb 11 11:44:09 test-bm2 magnum-conductor[2282404]: ERROR magnum.conductor.handlers.common.trust_manager magnum.common.exception.TrustCreateFailed: Failed to create trust for trustee 81b0e24a4d82446eb679433b2b004641.
Feb 11 11:44:09 test-bm2 magnum-conductor[2282404]: ERROR magnum.conductor.handlers.common.trust_manager 
Feb 11 11:44:09 test-bm2 magnum-conductor[2282404]: ERROR oslo_messaging.rpc.server [None req-b780c8d2-0194-42a4-82f6-614da8c9689b demo demo] Exception during message handling: magnum.common.exception.TrusteeOrTrustToClusterFailed: Failed to create trustee or trust for Cluster: 8de65e9d-8c3b-436e-841f-761926eaaab5

What version of devstack/openstack are you running in the CI? Using main branches seems to not work even for basic functionality at the moment.

[gabriel-samfira]

Hi @mnaser

There were 2 issues:

• The flatcar image seems to have an invalid GPT partition table according to Nova. This was a change in nova that validates the image. We can work around it
• The v1.27.15 flatcar image does not exist, but curl was lacking the --fail flag, so it downloaded a file with the text NoSuckKey and uploaded that to glance.

I think the latest available image for flatcar is v1.27.4. Switching to that made things work in my local env. Could we build a new image for Flatcar?

[gabriel-samfira]

the rocky tests seem to fail due to the same reason (image URL returns 404)