Error 5007 when Backing up VMs to a DataDomain Repository
Recently I have deployed my first DELLEMC Data Domain 2200 appliance to be used as a Backup Repository with Veeam Backup and Replication 9.5 Enterprise Edition.
This edition is required to leavarge Data Domain Boost (DDBOOST) which is licensed also on the appliance.
I followed the Veeam EMC Data Domain Storage with Veeam Backup & Replication: Configuration Best Practices and Performance Expectations to configure the backup repository and the backup jobs to get the best out of the setup.
All was going well until one day I noticed that backups were failing with the following error:
| Error: Failed to call RPC function ‘DDBoostFcIsExist’: operation is not implemented. Err: 5007. Failed to connect to storage. Storage: |
Rescanning the Backup Repository also failed with a similar error
Opening the properties window of the repository brought up the error : operation is not implemented. Err: 5007
I tried looking around the internet to see what might have caused this but I could not find any information about the error but I came across an EMC article “How to troubleshoot DataDomain DDBoost connectivity and performance” which in turn links you to a document suggesting the use of a command line tool, ddpconnchk.exe, to test the connectivity to the appliance.
So I downloaded the tool and ran the commands required to test the connectivity.
The output was similar to the below:
C:\ddpconncheck> ddpconnchk.exe -s 192.168.x.x -u ddboostuser -p ddboostpass DDP CLIENT LIBRARY VERSION 3:3:0:4-550644 SERVER: 192.168.x.x ---------------------- *** CONNECT SERVER TEST, 192.168.x.x DDLOG: ERR: [3198:3C24] ddpi_connect_with_user_pwd() failed, Hostname: 192.168.x.x, Err: 5007-nfs ddp getinfo failed (nfs: Operation not supported) connect_server_user_pwd() FAILED, DDP error 5007 (operation is not implemented) *** Retry/diagnose, connect_server_user_pwd()/log all msgs *** DDLOG: DBG: [3198:3C24] ddcl_vrapid_get_host_saddr: host_name 192.168.x.x has host_ip 192.168.x.x and NFS port 2049 found DDLOG: INFO: [3198:3C24] NFS connect on host ip=192.168.x.x DDLOG: DBG: [3198:3C24] clnt_async_tcp_connect: attempting to connect() on port =2049 ip=192.168.x.x DDLOG: INFO: [3198:3C24] 192.168.x.x is an IP string, can't look for failover DDLOG: INFO: [3198:3C24] number of sslquery 2 DDLOG: INFO: [3198:3C24] number of ssl_query_success = 2 DDLOG: INFO: [3198:3C24] DDBoost OST_SSL_QUERY success with auth_mode:0, recover:0 DDLOG: DBG: [3198:3C24] ddcl_nfs_ost_connect: MOUNT information over NFS port DDLOG: DBG: [3198:3C24] ddcl_nfs_ost_update_auth: Updated RPC AUTH with uid=0 gid=0 DDLOG: DBG: [3198:3C24] Permission passed on retry 0 DDLOG: DBG: [3198:3C24] ddcl_nfs_ost_update_auth: Updated RPC AUTH with uid=501 gid=100 DDLOG: INFO: [3198:3C24] clnt_async_destroy: RPC_CANTRECV will fail all the pending jobs and close socket DDLOG: ERR: [3198:3C24] ddpi_connect_with_user_pwd() failed, Hostname: 192.168.x.x, Err: 5007-nfs ddp getinfo failed (nfs: Operation not supported) connect_server_user_pwd()/log all msgs DONE ******************************************************** ** DDP connect FAILED ** Servername: 192.168.x.x ** Username: ddboostuser ** Password: ************* ** Error: operation is not implemented ** ** - Verify OST is licensed and enabled on the server ** - Verify username/passwd matches values ** configured on server ** - Verify that access by this host is allowed ******************************************************** DDP Connect Server Test FAILED |
From the output I noticed (nfs: Operation not supported). I checked if the services were started on the appliance and indeed they were disabled.
I then remembered that I had disabled them because I was trying to secure the system by disabling services that I was not using. I was only using CIFS and DDBoost so I thought NFS was not required. The next day I found out I was very wrong. Although I still cannot find the link between NFS and DDBoost, I had to enable it for everything to return to normal.
Running a ddpconnchk after enabling NFS gave a very much different and positive result:
DDP CLIENT LIBRARY VERSION 3:3:0:4-550644 SERVER: 192.168.x.x ---------------------- *** CONNECT SERVER TEST, 192.168.x.x DDP Connect Server Test PASSED *** LIST SUs TEST, 192.168.x.x 1 SUs total List SUs Test PASSED No further tests possible. Specify -l to enable more tests SUCCESS: All tests completed |
Hopefully someone else finds it useful and will not spend much time into resolving this error.
Let us know in the comments below
Great blog post, and you saved us a few hours !
Thank you. Glad it Helped you! 🙂
Thank you!!!!
You’re Welcome! 🙂