Recently I got a requirement to implement encryption for NFS traffic.
NFS communication is clear text based communication. Transfer of confidential data should be encrypted between the client and NFS server. Let me jump into the step by step implementation of this configuration.
I have used CentOS7 for the POC environment. Same steps would work with RHEL7 as well.
A pictorial representation of stunnel communication has given under Appendix3. Readers who wish to understand the communication flow might jump there.
LAB environment: 2 X Virtualbox vm: 1 vCPU/1GB RAM
- NFS server name: nfsserver.lab.com
IP address: 192.168.0.112
2. NFS client name: nfsclient.lab.com
IP address: 192.168.0.114
Steps to be executed on NFS server
- Set hostname on NFS server.
# hostnamectl set-hostname nfsserver.lab.com
- Set SELINUX to permissive.
# setenforce 0
- Stop firewalld
# systemctl stop firewalld
Note: These steps were executed temporarily. Once I manage to get this work with SELINUX and Firewall I would be updating this document.
- Append the following entries in /etc/hosts.
192.168.0.112 nfsserver nfsserver.lab.com
192.168.0.114 nfsclient nfsclient.lab.com
- Install NFS server and required packages.
# yum -y install nfs-utils
- Enable and start services related to NFS server.
# for i in rpcbind nfs-server nfs-idmap nfs-lock nfs-idmap
do
systemctl enable $i; systemctl start $i
done
- Create a path to be shared over NFS
# mkdir /home/share
- Configure and export the shares as mentioned below.
# cat /etc/exports
/home/share 127.0.0.1(fsid=0,ro,insecure)
Note: Loopback entry looks little weird I know but this is required for this setup. insecure option is also required to accept connections from unprivileged ports.
# exportfs -av
Let us setup Stunnel now.
Install stunnel package.
# yum -y install stunnel
- Next, create a user and directories to run the software - on this platform the RPM package does not create the user or directories:
# useradd -r -m -d /var/run/stunnel -s /bin/false stunnel
- The tmpfiles.d configuration to recreate the directory on reboot:
# echo "d /var/run/stunnel 0770 stunnel stunnel -" > /etc/tmpfiles.d/stunnel.conf
- Create the systemd unit file to run stunnel as a service:
[root@nfsclient ~]# cat << EOF > /etc/systemd/system/stunnel.service
[Unit]
Description=SSL tunnel for network daemons
After=syslog.target
[Service]
ExecStart=/usr/bin/stunnel
Type=forking
[Install]
WantedBy=multi-user.target
EOF
# openssl req -new -newkey rsa:2048 -days 3650 \
-nodes -x509 -sha256 \
-subj '/CN=127.0.0.1/O=localhost/C=US' \
-keyout /etc/stunnel/stunnel.pem \
-out /etc/stunnel/stunnel.pem
# chmod 400 /etc/stunnel/stunnel.pem
Next, create the stunnel server oriented config file; in our example we're using NFS service so we'll choose the ports accordingly to have stunnel accept the connection on the IP port 2363, then pass the connection to the localhost port 2049:
[root@nfsclient ~]# cat << EOF > /etc/stunnel/stunnel.conf
chroot = /var/run/stunnel
setuid = stunnel
setgid = stunnel
pid = /stunnel.pid
fips = no
[nfs_server]
client = no
accept = nfsserver.lab.com:2363
connect = 127.0.0.1:2049
cert = /etc/stunnel/stunnel.pem
key = /etc/stunnel/stunnel.pem
# stunnel 4.53 (Ubuntu 14) only supports TSLv1 not TLSv1.2
# stunnel 4.56 (CentOS 7) supports both TLSv1 and TSLv1.2
sslVersion = TLSv1
EOF
- Start & enable stunnel service
# systemctl start stunnel
# systemctl enable stunnel
Steps specific to NFS Client
- Set hostname on NFS Client.
# hostnamectl set-hostname nfsclient.lab.com
- Ensure host entries are updated in /etc/hosts file.
- Set SELINUX to permissive.
# setenforce 0
- Stop firewalld
# systemctl stop firewalld
Note: These steps were executed temporarily. Once I manage to get this work with SELINUX and Firewall I would be updating this document.
- Append the following entries in /etc/hosts.
192.168.0.112 nfsserver nfsserver.lab.com
192.168.0.114 nfsclient nfsclient.lab.com
Stunnel setup on client end
# yum -y install stunnel
Next, create a user and directories to run the software - on this platform the RPM package does not create the user or directories:
# useradd -r -m -d /var/run/stunnel -s /bin/false stunnel
- The tmpfiles.d configuration to recreate the directory on reboot:
# echo "d /var/run/stunnel 0770 stunnel stunnel -" > /etc/tmpfiles.d/stunnel.conf
- Create the systemd unit file to run stunnel as a service:
# cat << EOF > /etc/systemd/system/stunnel.service
[Unit]
Description=SSL tunnel for network daemons
After=syslog.target
[Service]
ExecStart=/usr/bin/stunnel
Type=forking
[Install]
WantedBy=multi-user.target
EOF
The client does not require a SSL certificate; create the client oriented config file that accepts a connection on local port 2323 and talks to the remote stunnel on 2363:
# cat << EOF > /etc/stunnel/stunnel.conf
chroot = /var/run/stunnel
setuid = stunnel
setgid = stunnel
pid = /stunnel.pid
[nfs_client]
client = yes
accept = 127.0.0.1:2323
connect = nfsserver.lab.com:2363
# stunnel 4.53 (Ubuntu 14) only supports TSLv1 not TLSv1.2
# stunnel 4.56 (CentOS 7) supports both TLSv1 and TSLv1.2
sslVersion = TLSv1
EOF
# systemctl start stunnel
# systemctl enable stunnel
- Update /etc/fstab with the following entry.
[root@nfsclient ~]# grep share /etc/fstab
localhost:/ /home/sslmount nfs noauto,vers=4.2,proto=tcp,port=2323 0 0
- Create the mount point & mount the share. This will be mounted using stunnel. And the traffic would be encrypted between the client and NFS server.
# mkdir /home/sslmount
# mount /home/sslmount
Testing
Let us see the difference between an NFS mount with encryption and without encryption.
We have our NFS share mounted under /home/sslmount with encryption support. Let us mount the same share under /mnt without encryption for testing purpose.
Mount the share under /mnt ( without stunnel)We need to update the share export configuration on the server to achieve this.
- Update /etc/exports on nfsserver.lab.com with the following entry.
- /home/share nfsclient.lab.com(ro)
- Re-export the shares with the new configuration.
- # exportfs -rv
Mount the share on nfsclient.lab.com as mentioned below.
# mount -t nfs nfsserver.lab.com:/home/share /mnt
- Capture the traffic on NFS client on one terminal.
# tcpdump -A -c 200 port 2049
- Create a file under /home/share on NFS server with some content. Then we can read this file from the client end and generate some traffic. Sample Traffic is attached under Appendix1.
# cat /mnt/file1
Let us capture the traffic and see how the traffic look like under the stunnel configured share.
- Capture the traffic on NFS client on one terminal.
# tcpdump -A -c 200 port 2363
- Read file1 using the SSL supported mount path /home/sslmount/file1
# cat /home/sslmount/file1
Sample Traffic is attached as Appendix2.
Following documents had helped me to setup this POC. You may go through them if interested.
Reference:
- http://kb.ictbanking.net/article.php?id=388&oid=2
- https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/security_guide/sec-using_stunnel
- https://www.linuxjournal.com/content/encrypting-nfsv4-stunnel-tls
Appendices:
Appendix1:
09:30:10.254725 IP nfsclient.864 > nfsserver.nfs: Flags [P.], seq 4081390743:4081390847, ack 3818043196, win 259, options [nop,nop,TS val 5098426 ecr 5073639], length 104: NFS request xid 3398054478 100 access fh Un/0100010000000000 NFS_ACCESS_READ|NFS_ACCESS_LOOKUP|NFS_ACCESS_MODIFY|NFS_ACCESS_EXTEND|NFS_ACCESS_DELETE
E....l@.@......r...p.`...E.....<...........
.M...Mj....d..2N...........................,.A.!....nfsclient.lab.com...........................................
09:30:10.255466 IP nfsserver.nfs > nfsclient.864: Flags [P.], seq 1:125, ack 104, win 227, options [nop,nop,TS val 5097632 ecr 5098426], length 124: NFS reply xid 3398054478 reply ok 120 access c 001f
E....b@.@......p...r...`...<.E.............
.M...M.....x..2N....................................................... ...............................raa.]..hIaa.5....aa.!.o......
09:30:10.255481 IP nfsclient.864 > nfsserver.nfs: Flags [.], ack 125, win 259, options [nop,nop,TS val 5098426 ecr 5097632], length 0
E..4.m@.@..$...r...p.`...E...........Y.....
.M...M..
09:30:10.257665 IP nfsclient.864 > nfsserver.nfs: Flags [P.], seq 104:216, ack 125, win 259, options [nop,nop,TS val 5098428 ecr 5097632], length 112: NFS request xid 3414831694 108 getattr fh Unkno/01000181000000007500000000000000005AE099
E....n@.@......r...p.`...E.................
.M...M.....l..2N...........................,.A.!....nfsclient.lab.com.......................................u........Z..
09:30:10.258219 IP nfsserver.nfs > nfsclient.864: Flags [P.], seq 125:241, ack 216, win 227, options [nop,nop,TS val 5097635 ecr 5098428], length 116: NFS reply xid 3414831694 reply ok 112 getattr REG 644 ids 65534/65534 sz 27
E....c@.@......p...r...`.....E.o....24.....
.M...M.....p..2N...................................................................................uaa...mc.aa.|..L.aa.|..L.
09:30:10.259079 IP nfsclient.864 > nfsserver.nfs: Flags [P.], seq 216:332, ack 241, win 259, options [nop,nop,TS val 5098430 ecr 5097635], length 116: NFS request xid 3431608910 112 access fh Unkno/01000181000000007500000000000000005AE099 NFS_ACCESS_READ|NFS_ACCESS_MODIFY|NFS_ACCESS_EXTEND|NFS_ACCESS_EXECUTE
E....o@.@......r...p.`...E.o...,...........
.M...M.....p..2N...........................,.A.!....nfsclient.lab.com.......................................u........Z.....-
09:30:10.260372 IP nfsserver.nfs > nfsclient.864: Flags [P.], seq 241:365, ack 332, win 227, options [nop,nop,TS val 5097637 ecr 5098430], length 124: NFS reply xid 3431608910 reply ok 120 access c 0001
E....d@.@......p...r...`...,.E......06.....
.M...M.....x..2N.......................................................................................uaa...mc.aa.|..L.aa.|..L.....
09:30:10.300091 IP nfsclient.864 > nfsserver.nfs: Flags [.], ack 365, win 259, options [nop,nop,TS val 5098471 ecr 5097637], length 0
E..4.p@.@..!...r...p.`...E...........Y.....
.M...M..
Appendix2:
09:29:25.173066 IP nfsclient.57186 > nfsserver.mediacntrlnfsd: Flags [P.], seq 3181805598:3181805848, ack 4011792125, win 529, options [nop,nop,TS val 5053344 ecr 5040935], length 250
E.....@.@......r...p.b ;.............S.....
.M...L.'.... ..r.._.2...O&.|..)..g..U.....f..........1....7..|.c/ =A..%.
.O..CX.....n.a.....(n...YA....../.(''.?....A&i...$.....x;./..i.U,......YB8*L..^/C..U.?)j..>.B.;..I.$....0iO9....&...K...2L..E0...&...;..b...v.*E..,?..._N..X.q=X..B/q&.s..0...I.zjTf,c6=.
09:29:25.173695 IP nfsserver.mediacntrlnfsd > nfsclient.57186: Flags [P.], seq 1:251, ack 250, win 361, options [nop,nop,TS val 5052550 ecr 5053344], length 250
E...i.@.@.M....p...r ;.b...........i.......
.M...M...... 1...e....[b.4f0o.K.K. {.2....7..........p...-....z.k{..<.j..&......4....).MT)[.i..I..F.Yny.+...[..#.NL\..,.d..?..b.NBRQq......O...j...Q.?... .E?..)d..J-.lQ...^..................W..A7e.bmht/.oA..2x..<*....F`pDr.#g..e.....['.sN)-.Ke......."..)..rk
09:29:25.173712 IP nfsclient.57186 > nfsserver.mediacntrlnfsd: Flags [.], ack 251, win 548, options [nop,nop,TS val 5053345 ecr 5052550], length 0
E..4..@.@......r...p.b ;...........$.Y.....
.M...M..
09:29:25.174855 IP nfsclient.57186 > nfsserver.mediacntrlnfsd: Flags [P.], seq 250:580, ack 251, win 548, options [nop,nop,TS val 5053346 ecr 5052550], length 330
E..~..@.@..{...r...p.b ;...........$.......
.M...M...... .m..N...&....Y.....V......C..i...... ..5+...&_s..+..
..%MEs..^..3.2....3.....U.c%.....].....
+..f.k+3..X....)S.~U...)...>....59...#..r.Y8..>(3...0-.Ub^=..4...S..n.<.!P.8.K.q...~gO.;..k.....e..%...i.Q@... y37..F,*.H.'...\........e.....!...=...*?..=xH.&XK...p.1..b....}.2.........Qxq.4.pk..53..8M....T=....Qju..7I......=fi8r..j
09:29:25.175575 IP nfsserver.mediacntrlnfsd > nfsclient.57186: Flags [P.], seq 251:725, ack 580, win 369, options [nop,nop,TS val 5052552 ecr 5053346], length 474
E...i.@.@.L....p...r ;.b.......b...q"......
.M...M...... .s.a...C.: 5...y.L.U..I..:A.5..r.......,y%...3o....5....P...Gw).0B....-.S.!w.IR..B.m..V.....C.M.I......r.C.H.|...Q.$p8"=.v..8R...0..`....x0..I..,.q9.34. {.@*..p.n.z....).......JG|.1[.O.l.r.K{....?..#-....v.3A..PbC.o.B.x..........X...Q.<a&..'..iKY/.........C?. ....s..+.....v.b/..*.h.Qr|..........M..>....R.....z.!^.../......92.^.....F!.f..e..Mr1...,...i.......@.E..P}S1v@^.L.?B..KV.....x...NB)#6..qx..n.i.A+.BJ]......V...B.Y.5....i.......FD2..J....{=..y.e..~r.R..=..)."
09:29:25.176932 IP nfsclient.57186 > nfsserver.mediacntrlnfsd: Flags [P.], seq 580:846, ack 725, win 568, options [nop,nop,TS val 5053348 ecr 5052552], length 266
E..>..@.@......r...p.b ;...b.......8.c.....
.M...M...... V.-5....Zi..K...../....f.......b...............j..r.]..X.F..+94...g.8.6..}..q....j.Y$.-..,.A..P...i...k._...~.....,.]U.#.#.ea<zB..i?..D.. ....f.
.......&W.......t..S...gv._b...W.L.a..]..O5=.z.N!..YT;e.RJ......e........,.."5!U1j..[.3.I-..[.......'*M;.......B@LI.
09:29:25.177580 IP nfsserver.mediacntrlnfsd > nfsclient.57186: Flags [P.], seq 725:927, ack 846, win 377, options [nop,nop,TS val 5052554 ecr 5053348], length 202
E...i.@.@.M....p...r ;.b.......l...y.......
.M...M...... q.B.U.{,..9.#K.u.bg..N..mWs..){U.................y...#.A...Aw..o....r.....4......y|........}.._.'(.9hv.&v.......*....^]Xc....yI....~5........9.x.$No..3...K..9^.&z?...9i.......J... .Us.{...z.}7..[A}
09:29:25.179154 IP nfsclient.57186 > nfsserver.mediacntrlnfsd: Flags [P.], seq 846:1112, ack 927, win 588, options [nop,nop,TS val 5053350 ecr 5052554], length 266
E..>..@.@......r...p.b ;...l.......L.c.....
.M...M...... ..iL.w}..Rrs;.|.8..1..9x.n..aoru..........@..#!Pk...All..u@/~m.hsb.W....P.X....m...(.@.e...(h..=g*.*..-..6......x.....{.j.g.jE......._).........I.&...RIo.Q..
.v...(....JU..\...X..a.T..=......Y..O.y....x.^G.x.sw..wZ...=tGO..q..U1eLD..YA~<Uc...p..}...(.^*..G..".]
09:29:25.179745 IP nfsserver.mediacntrlnfsd > nfsclient.57186: Flags [P.], seq 927:1113, ack 1112, win 386, options [nop,nop,TS val 5052556 ecr 5053350], length 186
E...i.@.@.N....p...r ;.b.......v...........
.M...M...... .B.P.t..Mi....[P.c3,.7P.:..$..4......7^....6...0.ty...U....E.M.%y.3.....%t........./.......b5(..5l..x..u3).Z...9F..Y9*.._y.)Ww...V.=...c.-....J.....X..G..
.. .DU....;E:....Va.......
09:29:25.219519 IP nfsclient.57186 > nfsserver.mediacntrlnfsd: Flags [.], ack 1113, win 608, options [nop,nop,TS val 5053391 ecr 5052556], length 0
E..4..@.@......r...p.b ;...v...U...`.Y.....
.M...M..
Appendix3:
Here I am trying to provide a pictorial representation of this setup.
An NFS request initiated by the NFS client would hit 2323 (accept entry under stunnel configuration on the client) port listening on loopback interface which is our stunnel service on the client. Stunnel would redirect this request on to the connect entry which is pointing at NFS server and its Stunnel port number,2363 (accept entry on NFS server side stunnel configuration. Stunnel service on NFS server would redirect this communication onto 2049 which is the connect entry on NFS server end Stunnel configuration.
Port 2049 is our NFS server port. Finally the request has reached NFS server.
No comments:
Post a Comment