[LTS/blocker] CLOUDSTACK-6432: Prevent DNS reflection attacks by yadvr · Pull Request #1663 · apache/cloudstack

yadvr · 2016-08-26T10:37:14Z

CLOUDSTACK-6432: Prevent DNS reflection attacks

DNS on VR should not be publically accessible as it may be prone to DNS
amplification/reflection attacks. This fixes the issue by only allowing VR
DNS (port 53) to be accessible from guest network cidr, as per the fix in:
https://issues.apache.org/jira/browse/CLOUDSTACK-6432

- Only allows guest network cidrs to query VR DNS on port 53.
- Includes marvin smoke test that checks the VR DNS accessibility checks from
  guest and non-guest network.
- Fixes Marvin sshClient to avoid using ssh agent when password is provided,
  previous some environments may have seen 'No existing session' exception without
  this fix.
- Adds a new dnspython dependency that is used to perform dns resolutions in the
  tests.

Due to repository commit issues I've created this PR, based on #1653 .

/cc @jburwell @karuturi @NuxRo @ustcweizhou @wido and others

yadvr · 2016-08-26T14:03:29Z

New smoke test result:

$ nosetests --with-xunit --xunit-file=integration-test-results.xml --with-marvin --marvin-config=/home/bhaisaab/Lab/apache/marvin-cfgs/adv-kvm.cfg -s -a tags=advanced,required_hardware=true --zone=KVM-advzone --hypervisor=KVM test_router_dns.py

==== Marvin Init Started ====

=== Marvin Parse Config Successful ===

=== Marvin Setting TestData Successful===

==== Log Folder Path: /tmp//MarvinLogs//Aug_26_2016_19_25_30_U977TH. All logs will be available here ====

=== Marvin Init Logging Successful===

==== Marvin Init Successful ====
Creating Admin Account for domain db10a868-6b7a-11e6-863f-a434d91cd37e on zone d7eb01c3-f5e4-4643-ae8a-14372778ffb4
Creating Service Offering on zone d7eb01c3-f5e4-4643-ae8a-14372778ffb4
Creating Network Offering on zone d7eb01c3-f5e4-4643-ae8a-14372778ffb4
Creating Network for Account test-a-TestRouterDns-64PRTV using offering 4a0b3e7e-c585-4b96-8040-736161597b62
Creating guest VM for Account test-a-TestRouterDns-64PRTV using offering 1e1db335-c2e0-48b2-a288-9adf88a10adb
Starting test_router_dns_externalips...
Querying VR DNS IP: 192.168.20.18
VR DNS query failed from non-guest network IP as expected
=== TestName: test_router_dns_externalipquery | Status : SUCCESS ===

Starting test_router_dns_guestipquery...
Creating Firewall rule for VM ID: b00b68b8-9445-4925-a631-f4bbd54bc6b9
Creating NAT rule for VM ID: b00b68b8-9445-4925-a631-f4bbd54bc6b9
SSH into guest VM with IP: 192.168.20.18
====Trying SSH Connection: Host:192.168.20.18 User:root Port:22 RetryCnt:8===
SshClient: Exception under createConnection: ['Traceback (most recent call last):\n', ' File "/usr/local/lib/python2.7/dist-packages/marvin/sshClient.py", line 122, in createConnection\n allow_agent=False)\n', ' File "/usr/local/lib/python2.7/dist-packages/paramiko/client.py", line 324, in connect\n raise NoValidConnectionsError(errors)\n', 'NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 192.168.20.18\n']
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/marvin/sshClient.py", line 122, in createConnection
allow_agent=False)
File "/usr/local/lib/python2.7/dist-packages/paramiko/client.py", line 324, in connect
raise NoValidConnectionsError(errors)
NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 192.168.20.18
====Trying SSH Connection: Host:192.168.20.18 User:root Port:22 RetryCnt:7===
SshClient: Exception under createConnection: ['Traceback (most recent call last):\n', ' File "/usr/local/lib/python2.7/dist-packages/marvin/sshClient.py", line 122, in createConnection\n allow_agent=False)\n', ' File "/usr/local/lib/python2.7/dist-packages/paramiko/client.py", line 324, in connect\n raise NoValidConnectionsError(errors)\n', 'NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 192.168.20.18\n']
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/marvin/sshClient.py", line 122, in createConnection
allow_agent=False)
File "/usr/local/lib/python2.7/dist-packages/paramiko/client.py", line 324, in connect
raise NoValidConnectionsError(errors)
NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 192.168.20.18
====Trying SSH Connection: Host:192.168.20.18 User:root Port:22 RetryCnt:6===
SshClient: Exception under createConnection: ['Traceback (most recent call last):\n', ' File "/usr/local/lib/python2.7/dist-packages/marvin/sshClient.py", line 122, in createConnection\n allow_agent=False)\n', ' File "/usr/local/lib/python2.7/dist-packages/paramiko/client.py", line 324, in connect\n raise NoValidConnectionsError(errors)\n', 'NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 192.168.20.18\n']
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/marvin/sshClient.py", line 122, in createConnection
allow_agent=False)
File "/usr/local/lib/python2.7/dist-packages/paramiko/client.py", line 324, in connect
raise NoValidConnectionsError(errors)
NoValidConnectionsError: [Errno None] Unable to connect to port 22 on 192.168.20.18

====Trying SSH Connection: Host:192.168.20.18 User:root Port:22 RetryCnt:5===
===SSH to Host 192.168.20.18 port : 22 SUCCESSFUL===
{Cmd: nslookup google.com via Host: 192.168.20.18} {returns: [u'Server:\t\t10.1.1.1', u'Address:\t10.1.1.1#53', u'', u'Non-authoritative answer:', u'Name:\tgoogle.com', u'Address: 216.58.199.142', u'']}
SSH nslookup result: [u'Server:\t\t10.1.1.1', u'Address:\t10.1.1.1#53', u'', u'Non-authoritative answer:', u'Name:\tgoogle.com', u'Address: 216.58.199.142', u'']
=== TestName: test_router_dns_guestipquery | Status : SUCCESS ===

jburwell · 2016-08-29T02:13:50Z

+        except Exception as e:
+            self.fail("Failed to SSH into VM - %s due to exception: %s" % (nat_rule1.ipaddress, e))
+
+        self.assertTrue(result is not None and "google.com" in result and "#53" in result,


Minor nit: Consider breaking up into three asserts with more detailed messages to improve the isolation of a failure

jburwell · 2016-08-29T02:22:51Z

@NuxRo Any thoughts to add?

yadvr · 2016-08-29T07:06:55Z

@jburwell @NuxRo thanks for the reviews, I've fixed the outstanding issues. Please re-review.

NuxRo · 2016-08-29T08:48:06Z

Thanks a lot for implementing this properly :)

borisstoyanov · 2016-08-30T13:20:36Z

LGTM, I was able to build and test the PR. Found a little issue with the tests and fixed it.
Here are the tests results:

$ nosetests --with-xunit --xunit-file=integration-test-results.xml --with-marvin --marvin-config=advanced_ccs.cfg -s -a tags=advanced,required_hardware=true --zone=zone1 --hypervisor=kvm cloudstack/test/integration/smoke/test_router_dns.py -vv
nose.config: INFO: Ignoring files matching ['^\\.', '^_', '^setup\\.py$']

==== Marvin Init Started ====

=== Marvin Parse Config Successful ===

=== Marvin Setting TestData Successful===

==== Log Folder Path: /tmp//MarvinLogs//Aug_30_2016_15_44_51_LNKTQ1. All logs will be available here ====

=== Marvin Init Logging Successful===

==== Marvin Init Successful ====
Creating Admin Account for domain b1376fae-6e2a-11e6-bca7-000c290e77f6 on zone 7060c2b9-7ea2-475f-9b74-56ce80444feb
Creating Service Offering on zone 7060c2b9-7ea2-475f-9b74-56ce80444feb
Creating Network Offering on zone 7060c2b9-7ea2-475f-9b74-56ce80444feb
Creating Network for Account test-a-TestRouterDns-CF5DZ4 using offering 8c206825-65e7-4aaa-9850-b8e804f523ef
Creating guest VM for Account test-a-TestRouterDns-CF5DZ4 using offering 6fd51c75-fe0f-4673-8b1a-a312fe5605c4
Starting test_router_dns_externalips...
Querying VR DNS IP: 192.168.1.103
VR DNS query failed from non-guest network IP as expected
=== TestName: test_router_dns_externalipquery | Status : SUCCESS ===

Starting test_router_dns_guestipquery...
Creating Firewall rule for VM ID: cf929b5d-4ce1-4c86-b389-5baba0a5d8e7
Creating NAT rule for VM ID: cf929b5d-4ce1-4c86-b389-5baba0a5d8e7
SSH into guest VM with IP: 192.168.1.103
====Trying SSH Connection: Host:192.168.1.103 User:root                                   Port:22 RetryCnt:8===
SshClient: Exception under createConnection: ['Traceback (most recent call last):\n', '  File "/usr/local/lib/python2.7/site-packages/marvin/sshClient.py", line 122, in createConnection\n    allow_agent=False)\n', '  File "/usr/local/lib/python2.7/site-packages/paramiko/client.py", line 305, in connect\n    retry_on_signal(lambda: sock.connect(addr))\n', '  File "/usr/local/lib/python2.7/site-packages/paramiko/util.py", line 269, in retry_on_signal\n    return function()\n', '  File "/usr/local/lib/python2.7/site-packages/paramiko/client.py", line 305, in <lambda>\n    retry_on_signal(lambda: sock.connect(addr))\n', '  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 228, in meth\n    return getattr(self._sock,name)(*args)\n', 'error: [Errno 51] Network is unreachable\n']
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/marvin/sshClient.py", line 122, in createConnection
    allow_agent=False)
  File "/usr/local/lib/python2.7/site-packages/paramiko/client.py", line 305, in connect
    retry_on_signal(lambda: sock.connect(addr))
  File "/usr/local/lib/python2.7/site-packages/paramiko/util.py", line 269, in retry_on_signal
    return function()
  File "/usr/local/lib/python2.7/site-packages/paramiko/client.py", line 305, in <lambda>
    retry_on_signal(lambda: sock.connect(addr))
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 228, in meth
    return getattr(self._sock,name)(*args)
error: [Errno 51] Network is unreachable
====Trying SSH Connection: Host:192.168.1.103 User:root                                   Port:22 RetryCnt:7===
SshClient: Exception under createConnection: ['Traceback (most recent call last):\n', '  File "/usr/local/lib/python2.7/site-packages/marvin/sshClient.py", line 122, in createConnection\n    allow_agent=False)\n', '  File "/usr/local/lib/python2.7/site-packages/paramiko/client.py", line 305, in connect\n    retry_on_signal(lambda: sock.connect(addr))\n', '  File "/usr/local/lib/python2.7/site-packages/paramiko/util.py", line 269, in retry_on_signal\n    return function()\n', '  File "/usr/local/lib/python2.7/site-packages/paramiko/client.py", line 305, in <lambda>\n    retry_on_signal(lambda: sock.connect(addr))\n', '  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 228, in meth\n    return getattr(self._sock,name)(*args)\n', 'error: [Errno 51] Network is unreachable\n']
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/marvin/sshClient.py", line 122, in createConnection
    allow_agent=False)
  File "/usr/local/lib/python2.7/site-packages/paramiko/client.py", line 305, in connect
    retry_on_signal(lambda: sock.connect(addr))
  File "/usr/local/lib/python2.7/site-packages/paramiko/util.py", line 269, in retry_on_signal
    return function()
  File "/usr/local/lib/python2.7/site-packages/paramiko/client.py", line 305, in <lambda>
    retry_on_signal(lambda: sock.connect(addr))
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 228, in meth
    return getattr(self._sock,name)(*args)
error: [Errno 51] Network is unreachable
====Trying SSH Connection: Host:192.168.1.103 User:root                                   Port:22 RetryCnt:6===
===SSH to Host 192.168.1.103 port : 22 SUCCESSFUL===
{Cmd: nslookup google.com via Host: 192.168.1.103} {returns: [u'Server:\t\t10.1.1.1', u'Address:\t10.1.1.1#53', u'', u'Non-authoritative answer:', u'Name:\tgoogle.com', u'Address: 212.39.82.187', u'Name:\tgoogle.com', u'Address: 212.39.82.174', u'Name:\tgoogle.com', u'Address: 212.39.82.152', u'Name:\tgoogle.com', u'Address: 212.39.82.180', u'Name:\tgoogle.com', u'Address: 212.39.82.167', u'Name:\tgoogle.com', u'Address: 212.39.82.181', u'Name:\tgoogle.com', u'Address: 212.39.82.159', u'Name:\tgoogle.com', u'Address: 212.39.82.153', u'Name:\tgoogle.com', u'Address: 212.39.82.146', u'Name:\tgoogle.com', u'Address: 212.39.82.173', u'Name:\tgoogle.com', u'Address: 212.39.82.166', u'Name:\tgoogle.com', u'Address: 212.39.82.160', u'']}
=== TestName: test_router_dns_guestipquery | Status : SUCCESS ===

===final results are now copied to: /tmp//MarvinLogs/test_router_dns_IG8LQP===

yadvr · 2016-08-30T14:52:14Z

Thanks @borisstoyanov

DNS on VR should not be publically accessible as it may be prone to DNS amplification/reflection attacks. This fixes the issue by only allowing VR DNS (port 53) to be accessible from guest network cidr, as per the fix in: https://issues.apache.org/jira/browse/CLOUDSTACK-6432 - Only allows guest network cidrs to query VR DNS on port 53. - Includes marvin smoke test that checks the VR DNS accessibility checks from guest and non-guest network. - Fixes Marvin sshClient to avoid using ssh agent when password is provided, previous some environments may have seen 'No existing session' exception without this fix. - Adds a new dnspython dependency that is used to perform dns resolutions in the tests. Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

yadvr · 2016-08-30T17:11:15Z

Thanks all, based on the test results and reviews from this PR with the marvin test and the original PR #1653 from where the main changes were taken, I'll go ahead and merge this now.

@jburwell

[LTS/blocker] CLOUDSTACK-6432: Prevent DNS reflection attacksCLOUDSTACK-6432: Prevent DNS reflection attacks DNS on VR should not be publically accessible as it may be prone to DNS amplification/reflection attacks. This fixes the issue by only allowing VR DNS (port 53) to be accessible from guest network cidr, as per the fix in: https://issues.apache.org/jira/browse/CLOUDSTACK-6432 - Only allows guest network cidrs to query VR DNS on port 53. - Includes marvin smoke test that checks the VR DNS accessibility checks from guest and non-guest network. - Fixes Marvin sshClient to avoid using ssh agent when password is provided, previous some environments may have seen 'No existing session' exception without this fix. - Adds a new dnspython dependency that is used to perform dns resolutions in the tests. Due to repository commit issues I've created this PR, based on #1653 . /cc @jburwell @karuturi @NuxRo @ustcweizhou @wido and others * pr/1663: CLOUDSTACK-6432: Prevent DNS reflection attacks Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

s-seitz · 2016-09-23T08:55:23Z

After patching our systemvm.iso and the respective routers, I've noticed the iptables rules changed as given in CsAddress.py. These rules don't get any packets since two identical (but unpatched) rules apply before. I found these in CsApp.py.

yadvr · 2016-09-23T10:27:26Z

@s-seitz the fix should work for newly deployed VRs, for existing VR there may be existing rules or chains blocking the intended fix. Are you getting issues or seeing the same behaviour with new VRs? If yes, can you send a fix. Thanks.

yadvr force-pushed the 4.9-dnsreflection-attack branch from 5adacc9 to 3f588c3 Compare August 26, 2016 14:03

yadvr closed this Aug 26, 2016

yadvr reopened this Aug 26, 2016

yadvr changed the title ~~CLOUDSTACK-6432: Prevent DNS reflection attacks~~ [LTS/blocker] CLOUDSTACK-6432: Prevent DNS reflection attacks Aug 26, 2016

jburwell reviewed Aug 29, 2016
View reviewed changes

yadvr force-pushed the 4.9-dnsreflection-attack branch from 3f588c3 to 56ad2c8 Compare August 29, 2016 07:07

yadvr closed this Aug 29, 2016

yadvr reopened this Aug 29, 2016

yadvr mentioned this pull request Aug 29, 2016

Prevent DNS reflection attacks #1653

Closed

borisstoyanov force-pushed the 4.9-dnsreflection-attack branch from 56ad2c8 to 9033200 Compare August 30, 2016 13:18

yadvr force-pushed the 4.9-dnsreflection-attack branch from 9033200 to 14504dc Compare August 30, 2016 17:10

asfgit merged commit 14504dc into apache:4.9 Aug 30, 2016

DaanHoogland deleted the 4.9-dnsreflection-attack branch March 11, 2024 10:40

weizhouapache mentioned this pull request Mar 12, 2024

server: update default value of setting 'expose.dns.externally' to 'false" #8776

Merged

13 tasks

Navigation Menu

Uh oh!

[LTS/blocker] CLOUDSTACK-6432: Prevent DNS reflection attacks - #1663

[LTS/blocker] CLOUDSTACK-6432: Prevent DNS reflection attacks#1663
asfgit merged 1 commit into
apache:4.9from
shapeblue:4.9-dnsreflection-attack

yadvr commented Aug 26, 2016 •

edited

Loading

Uh oh!

yadvr commented Aug 26, 2016

Uh oh!

jburwell Aug 29, 2016

Uh oh!

yadvr Aug 29, 2016

Uh oh!

jburwell commented Aug 29, 2016

Uh oh!

yadvr commented Aug 29, 2016

Uh oh!

NuxRo commented Aug 29, 2016

Uh oh!

borisstoyanov commented Aug 30, 2016

Uh oh!

yadvr commented Aug 30, 2016

Uh oh!

yadvr commented Aug 30, 2016

Uh oh!

s-seitz commented Sep 23, 2016

Uh oh!

yadvr commented Sep 23, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Uh oh!

Conversation

yadvr commented Aug 26, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yadvr commented Aug 26, 2016

Uh oh!

jburwell Aug 29, 2016

Choose a reason for hiding this comment

Uh oh!

yadvr Aug 29, 2016

Choose a reason for hiding this comment

Uh oh!

jburwell commented Aug 29, 2016

Uh oh!

yadvr commented Aug 29, 2016

Uh oh!

NuxRo commented Aug 29, 2016

Uh oh!

borisstoyanov commented Aug 30, 2016

Uh oh!

yadvr commented Aug 30, 2016

Uh oh!

yadvr commented Aug 30, 2016

Uh oh!

s-seitz commented Sep 23, 2016

Uh oh!

yadvr commented Sep 23, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

yadvr commented Aug 26, 2016 •

edited

Loading