Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bucket Policy upload fails due to GetBucketRequestPayment not being implemented (Ceph) #2447

Open
1 task done
sj14 opened this issue Jan 9, 2025 · 0 comments
Open
1 task done

Bucket Policy upload fails due to GetBucketRequestPayment not being implemented (Ceph) #2447

sj14 opened this issue Jan 9, 2025 · 0 comments
Labels
needs_verified Some one might want to take a look at this and reproduce it to confirm

Comments

@sj14
Copy link

sj14 commented Jan 9, 2025

Summary

When I try to upload a bucket policy to Hetzner Object Storage (Ceph), it fails due to GetBucketRequestPayment not being implemented. The module documentation states it should not fail when requester_pays is set to false, but this also doesn't seem to work for me. Am I missing something?

Issue Type

Bug Report

Component Name

s3_bucket

Ansible Version

$ ansible --version
ansible [core 2.18.1]
  config file = None
  configured module search path = ['/Users/simon/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /opt/homebrew/Cellar/ansible/11.1.0_1/libexec/lib/python3.13/site-packages/ansible
  ansible collection location = /Users/simon/.ansible/collections:/usr/share/ansible/collections
  executable location = /opt/homebrew/bin/ansible
  python version = 3.13.1 (main, Dec  3 2024, 17:59:52) [Clang 16.0.0 (clang-1600.0.26.4)] (/opt/homebrew/Cellar/ansible/11.1.0_1/libexec/bin/python)
  jinja version = 3.1.5
  libyaml = True

Collection Versions

$ ansible-galaxy collection list

# /Users/simon/.ansible/collections/ansible_collections
Collection                               Version
---------------------------------------- -------
amazon.aws                               9.1.0

# /opt/homebrew/Cellar/ansible/11.1.0_1/libexec/lib/python3.13/site-packages/ansible_collections
Collection                               Version
---------------------------------------- -------
amazon.aws                               9.0.0
ansible.netcommon                        7.1.0
ansible.posix                            1.6.2
ansible.utils                            5.1.2
ansible.windows                          2.5.0
arista.eos                               10.0.1
awx.awx                                  24.6.1
azure.azcollection                       3.1.0
check_point.mgmt                         6.2.1
chocolatey.chocolatey                    1.5.3
cisco.aci                                2.10.1
cisco.asa                                6.0.0
cisco.dnac                               6.25.0
cisco.intersight                         2.0.20
cisco.ios                                9.0.3
cisco.iosxr                              10.2.2
cisco.ise                                2.9.6
cisco.meraki                             2.18.3
cisco.mso                                2.9.0
cisco.nxos                               9.2.1
cisco.ucs                                1.14.0
cloud.common                             4.0.0
cloudscale_ch.cloud                      2.4.0
community.aws                            9.0.0
community.ciscosmb                       1.0.9
community.crypto                         2.22.3
community.digitalocean                   1.27.0
community.dns                            3.1.0
community.docker                         4.1.0
community.general                        10.1.0
community.grafana                        2.1.0
community.hashi_vault                    6.2.0
community.hrobot                         2.0.2
community.library_inventory_filtering_v1 1.0.2
community.libvirt                        1.3.0
community.mongodb                        1.7.8
community.mysql                          3.11.0
community.network                        5.1.0
community.okd                            4.0.0
community.postgresql                     3.9.0
community.proxysql                       1.6.0
community.rabbitmq                       1.3.0
community.routeros                       3.1.0
community.sap_libs                       1.4.2
community.sops                           2.0.0
community.vmware                         5.2.0
community.windows                        2.3.0
community.zabbix                         3.2.0
containers.podman                        1.16.2
cyberark.conjur                          1.3.1
cyberark.pas                             1.0.30
dellemc.enterprise_sonic                 2.5.1
dellemc.openmanage                       9.9.0
dellemc.powerflex                        2.5.0
dellemc.unity                            2.0.0
f5networks.f5_modules                    1.32.1
fortinet.fortimanager                    2.8.2
fortinet.fortios                         2.3.8
google.cloud                             1.4.1
grafana.grafana                          5.6.0
hetzner.hcloud                           4.2.2
ibm.qradar                               4.0.0
ibm.spectrum_virtualize                  2.0.0
ibm.storage_virtualize                   2.5.0
ieisystem.inmanage                       3.0.0
infinidat.infinibox                      1.4.5
infoblox.nios_modules                    1.7.1
inspur.ispim                             2.2.3
junipernetworks.junos                    9.1.0
kaytus.ksmanage                          2.0.0
kubernetes.core                          5.0.0
kubevirt.core                            2.1.0
lowlydba.sqlserver                       2.3.4
microsoft.ad                             1.7.1
netapp.cloudmanager                      21.24.0
netapp.ontap                             22.13.0
netapp.storagegrid                       21.13.0
netapp_eseries.santricity                1.4.1
netbox.netbox                            3.20.0
ngine_io.cloudstack                      2.5.0
openstack.cloud                          2.3.0
ovirt.ovirt                              3.2.0
purestorage.flasharray                   1.32.0
purestorage.flashblade                   1.19.1
sensu.sensu_go                           1.14.0
splunk.es                                4.0.0
telekom_mms.icinga_director              2.2.1
theforeman.foreman                       4.2.0
vmware.vmware                            1.7.1
vmware.vmware_rest                       4.3.0
vultr.cloud                              1.13.0
vyos.vyos                                5.0.0
wti.remote                               1.0.10

AWS SDK versions

$ pip3 show boto boto3 botocore
WARNING: Package(s) not found: boto, boto3, botocore

Configuration

$ ansible-config dump --only-changed
CONFIG_FILE() = None
PAGER(env: PAGER) = less

GALAXY_SERVERS:

OS / Environment

MacOS 15.2 (arm64)

Steps to Reproduce

- name: s3
  hosts: localhost
  connection: local
  gather_facts: false
  tasks:
    - amazon.aws.s3_bucket:
        aws_access_key: "<REMOVED>"
        aws_secret_key: "<REMOVED>"
        endpoint_url: https://fsn1.your-objectstorage.com
        region: fsn1
        name: <bucket-name>
        state: present
        policy: "<anything>"
        debug_botocore_endpoint_logs: true
        ceph: true
        requester_pays: false
$ ansible-playbook -vvv s3.yaml

Expected Results

The playbook should be applied successfully

Actual Results

ansible-playbook [core 2.18.1]
  config file = None
  configured module search path = ['/Users/simon/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /opt/homebrew/Cellar/ansible/11.1.0_1/libexec/lib/python3.13/site-packages/ansible
  ansible collection location = /Users/simon/.ansible/collections:/usr/share/ansible/collections
  executable location = /opt/homebrew/bin/ansible-playbook
  python version = 3.13.1 (main, Dec  3 2024, 17:59:52) [Clang 16.0.0 (clang-1600.0.26.4)] (/opt/homebrew/Cellar/ansible/11.1.0_1/libexec/bin/python)
  jinja version = 3.1.5
  libyaml = True
No config file found; using defaults
host_list declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
script declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
auto declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
yaml declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
ini declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
toml declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Skipping callback 'default', as we already have a stdout callback.
Skipping callback 'minimal', as we already have a stdout callback.
Skipping callback 'oneline', as we already have a stdout callback.

PLAYBOOK: s3.yaml **************************************************************
1 plays in s3.yaml

PLAY [s3] **********************************************************************

TASK [amazon.aws.s3_bucket] ****************************************************
task path: /Users/simon/Desktop/ansible/s3.yaml:6
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: simon
<127.0.0.1> EXEC /bin/sh -c 'echo ~simon && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /Users/simon/.ansible/tmp `"&& mkdir "` echo /Users/simon/.ansible/tmp/ansible-tmp-1736430222.468454-36778-102655395157911 `" && echo ansible-tmp-1736430222.468454-36778-102655395157911="` echo /Users/simon/.ansible/tmp/ansible-tmp-1736430222.468454-36778-102655395157911 `" ) && sleep 0'
Using module file /Users/simon/.ansible/collections/ansible_collections/amazon/aws/plugins/modules/s3_bucket.py
<127.0.0.1> PUT /Users/simon/.ansible/tmp/ansible-local-36774dk50z9fy/tmpha6ndb0t TO /Users/simon/.ansible/tmp/ansible-tmp-1736430222.468454-36778-102655395157911/AnsiballZ_s3_bucket.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /Users/simon/.ansible/tmp/ansible-tmp-1736430222.468454-36778-102655395157911/ /Users/simon/.ansible/tmp/ansible-tmp-1736430222.468454-36778-102655395157911/AnsiballZ_s3_bucket.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '/opt/homebrew/Cellar/ansible/11.1.0_1/libexec/bin/python /Users/simon/.ansible/tmp/ansible-tmp-1736430222.468454-36778-102655395157911/AnsiballZ_s3_bucket.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'rm -f -r /Users/simon/.ansible/tmp/ansible-tmp-1736430222.468454-36778-102655395157911/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "/var/folders/68/c0sn4zgx5nqdyy8ll80850cm0000gn/T/ansible_amazon.aws.s3_bucket_payload_3ozacv4t/ansible_amazon.aws.s3_bucket_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_bucket.py", line 643, in handle_bucket_requester_pays
    requester_pays_status = get_bucket_request_payment(s3_client, name)
  File "/var/folders/68/c0sn4zgx5nqdyy8ll80850cm0000gn/T/ansible_amazon.aws.s3_bucket_payload_3ozacv4t/ansible_amazon.aws.s3_bucket_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py", line 119, in _retry_wrapper
    return _retry_func(
        func=partial_func,
    ...<5 lines>...
        base_class=cls.base_class,
    )
  File "/var/folders/68/c0sn4zgx5nqdyy8ll80850cm0000gn/T/ansible_amazon.aws.s3_bucket_payload_3ozacv4t/ansible_amazon.aws.s3_bucket_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/cloud.py", line 68, in _retry_func
    return func()
  File "/var/folders/68/c0sn4zgx5nqdyy8ll80850cm0000gn/T/ansible_amazon.aws.s3_bucket_payload_3ozacv4t/ansible_amazon.aws.s3_bucket_payload.zip/ansible_collections/amazon/aws/plugins/modules/s3_bucket.py", line 1565, in get_bucket_request_payment
    return s3_client.get_bucket_request_payment(Bucket=bucket_name).get("Payer")
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/var/folders/68/c0sn4zgx5nqdyy8ll80850cm0000gn/T/ansible_amazon.aws.s3_bucket_payload_3ozacv4t/ansible_amazon.aws.s3_bucket_payload.zip/ansible_collections/amazon/aws/plugins/module_utils/retries.py", line 107, in deciding_wrapper
    return unwrapped(*args, **kwargs)
  File "/opt/homebrew/Cellar/ansible/11.1.0_1/libexec/lib/python3.13/site-packages/botocore/client.py", line 569, in _api_call
    return self._make_api_call(operation_name, kwargs)
           ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/ansible/11.1.0_1/libexec/lib/python3.13/site-packages/botocore/client.py", line 1023, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (501) when calling the GetBucketRequestPayment operation: Not Implemented
fatal: [localhost]: FAILED! => {
    "boto3_version": "1.35.87",
    "botocore_version": "1.35.87",
    "changed": false,
    "error": {
        "code": "501",
        "message": "Not Implemented"
    },
    "invocation": {
        "module_args": {
            "accelerate_enabled": null,
            "access_key": "<REMOVED>",
            "acl": null,
            "aws_access_key": "<REMOVED>",
            "aws_ca_bundle": null,
            "aws_config": null,
            "aws_secret_key": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "bucket_key_enabled": null,
            "ceph": true,
            "debug_botocore_endpoint_logs": true,
            "delete_object_ownership": false,
            "delete_public_access": false,
            "dualstack": false,
            "encryption": null,
            "encryption_key_id": null,
            "endpoint_url": "https://fsn1.your-objectstorage.com",
            "force": false,
            "inventory": null,
            "name": "<REMOVED>",
            "object_lock_default_retention": null,
            "object_lock_enabled": null,
            "object_ownership": null,
            "policy": "<anything>",
            "profile": null,
            "public_access": null,
            "purge_tags": true,
            "region": "fsn1",
            "requester_pays": false,
            "secret_key": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "session_token": null,
            "state": "present",
            "tags": null,
            "validate_bucket_name": true,
            "validate_certs": true,
            "versioning": null
        }
    },
    "msg": "Failed to get bucket request payment: An error occurred (501) when calling the GetBucketRequestPayment operation: Not Implemented",
    "resource_actions": [
        "fsn1:GetBucketRequestPayment",
        "fsn1:HeadBucket",
        "fsn1:GetBucketVersioning"
    ],
    "response_metadata": {
        "http_headers": {
            "cache-control": "no-cache",
            "content-length": "136",
            "content-type": "text/html"
        },
        "http_status_code": 501,
        "retry_attempts": 0
    }
}

PLAY RECAP *********************************************************************
localhost                  : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

Code of Conduct

  • I agree to follow the Ansible Code of Conduct
@abikouo abikouo added needs_verified Some one might want to take a look at this and reproduce it to confirm and removed needs_triage labels Jan 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
needs_verified Some one might want to take a look at this and reproduce it to confirm
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@sj14 @abikouo