Skip to content

Add SQS & SNS integration tests#2505

Open
rlaunch wants to merge 21 commits intocelery:mainfrom
rlaunch:main
Open

Add SQS & SNS integration tests#2505
rlaunch wants to merge 21 commits intocelery:mainfrom
rlaunch:main

Conversation

@rlaunch
Copy link
Copy Markdown
Contributor

@rlaunch rlaunch commented Apr 11, 2026

This PR adds basic integration tests for the Amazon SQS & SNS transports as requested on PR #2479.

To achieve this I looked at using LocalStack, however an API key/licence was required and would have been a bit overkill for the testing needs of this project. I have used a cool project called GoAws, which runs as a Docker container and easily integrates as a drop-in replacement for AWS SQS and SNS, including testing Fanout (SNS -> SQS subscriptions).

A small fix was also made to how parameters are handled in the AsyncSQSConnection.make_request method, as it appears it should be able to handle a None value for the protocol_params parameter, which caused an exception. The small fix now correctly handles None values being passed, and tests have been added for validation.

@codecov
Copy link
Copy Markdown

codecov bot commented Apr 11, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.45%. Comparing base (9022fcc) to head (0037a04).
⚠️ Report is 2 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2505   +/-   ##
=======================================
  Coverage   82.44%   82.45%           
=======================================
  Files          79       79           
  Lines       10142    10146    +4     
  Branches     1165     1165           
=======================================
+ Hits         8362     8366    +4     
  Misses       1579     1579           
  Partials      201      201           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@auvipy auvipy requested review from auvipy and Copilot April 11, 2026 16:00
@auvipy auvipy added this to the 5.7.0 milestone Apr 11, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds SQS/SNS integration coverage (using GoAws) and fixes a None handling bug in the async SQS request parameter merge logic, aligning with prior work to prevent fanout-related regressions.

Changes:

  • Added a new tox integration environment for SQS backed by a GoAws Docker container.
  • Added SQS integration tests (including SNS->SQS fanout) and a unit test validating protocol_params=None handling.
  • Refactored AsyncSQSConnection.make_request parameter merging into a helper that safely handles None.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File Description
tox.ini Adds integration-sqs tox env and a goaws docker service definition.
kombu/asynchronous/aws/sqs/connection.py Fixes make_request param merging to tolerate protocol_params=None via _build_make_request_params.
t/unit/asynchronous/aws/sqs/test_connection.py Adds unit coverage for the new param-building helper behavior.
t/integration/test_sqs.py Introduces GoAws-backed integration tests for SQS, including fanout behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tox.ini Outdated
Comment thread t/unit/asynchronous/aws/sqs/test_connection.py Outdated
Comment thread t/unit/asynchronous/aws/sqs/test_connection.py Outdated
auvipy and others added 2 commits April 11, 2026 22:08
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 6 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread kombu/asynchronous/aws/sqs/connection.py Outdated
Comment thread t/unit/asynchronous/aws/sqs/test_connection.py Outdated
Comment thread t/integration/test_sqs.py Outdated
Comment thread t/integration/test_sqs.py Outdated
Comment thread t/integration/test_sqs.py Outdated
Comment thread t/integration/test_sqs.py
auvipy and others added 7 commits April 11, 2026 22:39
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
… with not using 'is_secure' in transport options config.
@auvipy
Copy link
Copy Markdown
Member

auvipy commented Apr 12, 2026

probably some suggested code did not work so integration tests are failing

@rlaunch
Copy link
Copy Markdown
Contributor Author

rlaunch commented Apr 12, 2026

Hey @auvipy,

Thanks for the quick review and feedback.
It seems the issue is due to the SNS module naming conflicting the SNS class being imported into sqs/__init__.py. This obfuscates the SNS module and makes importing it or patching is impossible.

The least intrusive change I could think of was to import the SNS class as SnsFanout, which prevents the naming collision.

Best,
Ryan

@auvipy
Copy link
Copy Markdown
Member

auvipy commented Apr 13, 2026

botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "http://localhost:4100/"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants