I was trying to make backup to a the vcenter vm with Veeam B&R 6.5 last patch, and i got the followed error: timed out waiting for all VDDK disks to close, so i tried to fix the error restarting managment agents on the host, consolidating the disk on the vm but nothing help.

So, i tried to clone the vm to make sure no vmdk are damage or are not working properly but when i try to do it, i get the error: error caused by file vm/vm-000002.vmdk, i can´t find that file browsing the datastore.

What should I do?? We are working with vSphere 5.0

Hi

I've been having trouble with the VDP Appliance in Version 6.1. Its a fresh install, running on a Windows vCenter 6.0.0 Build 3018524 and using a Buffalo TeraStation NAS as backup destination.

The problem starts when I try any kind of backup job. Before that everything works as intended. Creating jobs, managing the appliance etc. Once I start a backup job the appliance kind of freezes. In the web client it wont load any data

Sometimes in runs into the error that the time snyc. isnt working properly. Although I have checked vCenter Server, the hosts and the appliance. They're all configured with the proper ntp server and syncing properly. The VDP appliance is also showing the correct time. Restartet the tools services aswell.

At some point the Web Client looses the connection to the appliance.

Connecting over SSH to the appliance sometimes runs into timeouts aswell. Restarting the VDP appliance takes about 20-30 minutes.

UPDATE:

After literally a hour I was able to connect back into the appliance within the the web client. My test backup job started gradually counting up to 38% (Its 20GB small and really shouldnt take a hour to backup). After a few minutes it disconnected again.. the task stopping at 38%.

One thing to note: The VDPs virtual install disk is running on the same datastore as where its backing up to. Could this be the problem im having? I cant really any other problems within the config.

David

Hello Everyone,

Often in the morning we cannot connect to the VDP appliance from Vcenter. When I logon to the configuration page the internal proxy gives the following error:

"Appliance is currently not configured with any proxies. click on the action menu to configure a proxy"

Often the only way to get the internal proxy running is to restart the whole appliance.

This happens almost every morning.

The VDP appliance is running on a NFS datastore and is running version 6.1.1.19.

The performance test is passed.

Also getting the mesage "the most recent checkpoint for the vdp appliance was outdated"

Any idea how to solve this?

Hi,

We have a linux server running CentOS 6.2 that crashes each time the VMDR backup runs.
This has occurred 3 times now is becoming a concern.

When checking the syslog this message is displayed:

Apr 17 23:43:50 myVM kernel: sd 2:0:1:0: [sdb] Unhandled error code
Apr 17 23:43:50 myVM kernel: sd 2:0:1:0: [sdb] Result: hostbyte=DID_ABORT driverbyte=DRIVER_OK
Apr 17 23:43:50 myVM kernel: sd 2:0:1:0: [sdb] CDB: Write(10): 2a 00 02 85 86 e8 00 00 58 00
Apr 17 23:43:50 myVM kernel: end_request: I/O error, dev sdb, sector 42305256

The VMDR log shows:

4/17/2012 11:17:40 PM: Normal backup using Backup Job 1
4/17/2012 11:18:49 PM: Copying myVM
4/17/2012 11:20:15 PM: Performing incremental back up of disk "[Datastore] myVM/myVM-flat.vmdk" using "SCSI Hot-Add"
4/17/2012 11:46:51 PM: Trouble reading files, error -3942 ( delete snapshot failed)
4/17/2012 11:46:55 PM: 1 task errors
4/17/2012 11:46:55 PM: Completed: 5 files, 10,1 GB
$[//]4/17/2012 9:46:55 PM: Performance: 382.8 MB/minute$[//37/%D %.1T: Performance: %.1~F MB/minute]$[/D/30/2012-04-17T21:46:55.000Z+00:00/]$[/T/30/2012-04-17T21:46:55.000Z+00:00/]$[/F/5/382.8/]
4/17/2012 11:46:55 PM: Duration: 00:29:14 (00:02:29 idle/loading/preparing)

I have also found that the server gets pretty slow with high load as a result when the backup runs.
We also have other CentOS 6.2 servers and none of these have this problem.

Manual snapshots of the VM doesn´t cause this problem.

VMDR Appliance version 1.2.1.1616 , VMware ESX 4.1.

Thanks for any help and pointers on where to begin troubleshoot this problem ...

--------------

EDIT:

Problem was due to a process generating a lot of disk I/O and this together with the "Quisce guest file system" feature for vmware-tools made the VM OS hit the 120 second limit for waiting for disk write.

Hi All,

I have two VDP 6 instances. One week ago a saw that CPU usage on one of them  is almost always 100%. The backup works, Second instance have about 20% CPU utilization.

The diffrence I find: (second instance does not have this kind of errors)

root@clvdp01:/space/avamar/var/mc/server_log/#: tail -f axis2.log

2015-05-18 10:04:04,273 [218184632@qtp-1305303392-4784] ERROR org.apache.axis2.transport.http.AxisServlet  - Java heap space

java.lang.OutOfMemoryError: Java heap space

2015-05-18 10:04:37,105 [1384024202@qtp-1305303392-4789] ERROR org.apache.axis2.transport.http.AxisServlet  - Java heap space

java.lang.OutOfMemoryError: Java heap space

2015-05-18 10:05:13,130 [1795787615@qtp-1305303392-4779] ERROR org.apache.axis2.transport.http.AxisServlet  - Java heap space

java.lang.OutOfMemoryError: Java heap space

2015-05-18 10:05:49,640 [1593678748@qtp-1305303392-4821] ERROR org.apache.axis2.transport.http.AxisServlet  - Java heap space

java.lang.OutOfMemoryError: Java heap space

2015-05-18 10:09:11,437 [1107337480@qtp-1305303392-4777] ERROR org.apache.axis2.transport.http.AxisServlet  - Java heap space

java.lang.OutOfMemoryError: Java heap space

Below more information:

status.dpn

Mon May 18 10:05:20 CEST 2015  [clvdp01] Mon May 18 08:05:20 2015 UTC (Initialized Thu Oct  2 10:31:18 2014 UTC)

Node   IP Address     Version   State   Runlevel  Srvr+Root+User Dis Suspend Load UsedMB Errlen  %Full   Percent Full and Stripe Status by Disk

0.0     10.190.4.61 7.1.81-107  ONLINE fullaccess mhpu+0hpu+0hpu   1 false   6.05 7460  6866739  43.0%  43%(onl:1792) 43%(onl:1795) 43%(onl:1792)

Srvr+Root+User Modes = migrate + hfswriteable + persistwriteable + useraccntwriteable

System ID: 1412245878@00:50:56:A2:0D:56

All reported states=(ONLINE), runlevels=(fullaccess), modes=(mhpu+0hpu+0hpu)

System-Status: ok

Access-Status: full

Last checkpoint: cp.20150517093236 finished Sun May 17 11:33:38 2015 after 01m 01s (OK)

Last GC: finished Mon May 18 09:34:53 2015 after 01h 27m >> recovered 11.24 GB (OK)

Last hfscheck: finished Sun May 17 11:30:06 2015 after 01h 18m >> checked 2832 of 2832 stripes (OK)

Maintenance windows scheduler capacity profile is active.

  The maintenance window is currently running.

  Currently running task(s): crunchwait

  Next backup window start time: Tue May 19 00:00:00 2015 CEST

  Next maintenance window start time: Tue May 19 08:00:00 2015 CEST

dpnctl status

Identity added: /home/dpn/.ssh/dpnid (/home/dpn/.ssh/dpnid)

dpnctl: INFO: gsan status: up

dpnctl: INFO: MCS status: up.

dpnctl: INFO: Backup scheduler status: up.

dpnctl: INFO: axionfs status: down.

dpnctl: INFO: Maintenance windows scheduler status: enabled.

dpnctl: INFO: Unattended startup status: enabled.

dpnctl: INFO: [see log file "/usr/local/avamar/var/log/dpnctl.log"]

top

top - 10:11:05 up 3 days, 11:21,  1 user,  load average: 6.19, 5.49, 5.22

Tasks: 232 total,   1 running, 230 sleeping,   0 stopped,   1 zombie

Cpu(s): 52.5%us,  0.7%sy,  0.9%ni, 42.4%id,  3.6%wa,  0.0%hi,  0.1%si,  0.0%st

Mem:   7926796k total,  7515232k used,   411564k free,     9240k buffers

Swap:  4200988k total,        0k used,  4200988k free,  2833152k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND

8547 admin     20   0 2030m 1.3g 5580 S  596 16.9  19371:50 java

root@clvdp01:/space/avamar/var/mc/server_log/#: ps aux | grep 8547

admin     8547  387 16.8 2077128 1335904 ?     Sl   May14 19374:25 /usr/java/latest/bin/java -Xmx1G -XX:MaxPermSize=256m -XX:+HeapDumpOnOutOfMemoryError -server -ea -cp /usr/local/avamar/lib/avamar-mcsecurity.jar:/usr/local/avamar/lib/commons-cli-1.2.jar:/usr/local/avamar/lib/commons-lang-2.6.jar:/usr/local/avamar/lib/commons-logging-adapters-1.1.jar:/usr/local/avamar/lib/rsa-jsafe-cryptojFIPS-5.0.jar:/usr/local/avamar/lib/rsa-jsafe-util-5.0.jar:/usr/local/avamar/lib/spring-jdbc-3.1.1.RELEASE.jar:/usr/local/avamar/lib/spring-tx-3.1.1.RELEASE.jar:/usr/local/avamar/lib/mcserver.jar:/usr/local/avamar/var/mc/server_data/prefs:lib:/usr/local/avamar/lib/asn_server.jar:/usr/local/avamar/lib/mail.jar:/usr/local/avamar/lib/activation.jar:/usr/local/avamar/lib/xercesImpl.jar:/usr/local/avamar/lib/xmlsec.jar:/usr/local/avamar/lib/xmldsig.jar:lib/postgresql-8.4-702.jdbc4.jar:/usr/local/avamar/lib/jaxrpc.jar:/usr/local/avamar/lib/commons-logging-1.0.4.jar:/usr/local/avamar/lib/commons-validator.jar:/usr/local/avamar/lib/commons-discovery-0.2.jar:/usr/local/avamar/lib/commons-collections-3.2.1.jar:/usr/local/avamar/lib/saaj.jar:/usr/local/avamar/lib/wbem.jar:/usr/local/avamar/lib/wsdl4j-1.5.1.jar:/usr/local/avamar/lib/log4j-1.2.14.jar:/usr/local/avamar/lib/axis.jar:/usr/local/avamar/lib/vim25.jar:/usr/local/avamar/lib/commons-codec-1.3.jar:/usr/local/avamar/lib/jema_api.jar:/usr/local/avamar/lib/jetty-6.1.23.jar:/usr/local/avamar/lib/jetty-util-6.1.23.jar:/usr/local/avamar/lib/servlet-api-3.0.jar:/usr/local/avamar/lib/jsp-api-2.1.jar:/usr/local/avamar/lib/jsp-2.1.jar:/usr/local/avamar/lib/core-3.1.1.jar:/usr/local/avamar/lib/ant-1.6.5.jar:/usr/local/avamar/lib/apputils25.jar:/usr/local/avamar/lib/j2ssh-core-0.2.9.jar:/usr/local/avamar/lib/snmp4j-1.11.jar:/usr/local/avamar/lib/mccommons.jar:/usr/local/avamar/lib/user-authentication.jar:/usr/local/avamar/lib/nis-1.2.1.jar:/usr/local/avamar/lib/providerutil-1.2.1.jar:/usr/local/avamar/lib/commons-lang-2.6.jar:/usr/local/avamar/lib/commons-io-1.4.jar:/usr/local/avamar/lib/jutf7-1.0.0.jar:/usr/local/avamar/lib/i18n:/usr/local/avamar/lib/certj.jar:/usr/local/avamar/lib/cryptojFIPS.jar:/usr/local/avamar/lib/sslj.jar:/usr/local/avamar/lib/amqp-client-3.1.3.jar:/usr/local/avamar/lib/events-dispatcher-api-1.0.0.jar:/usr/local/avamar/lib/events-dispatcher-amqp-1.0.0.jar:/usr/local/avamar/lib/jackson-core-asl-1.8.3.jar:/usr/local/avamar/lib/jackson-mapper-asl-1.8.3.jar:/usr/local/avamar/lib/aopalliance-1.0.jar:/usr/local/avamar/lib/aspectjweaver-1.6.12.jar:/usr/local/avamar/lib/spring-aop-3.1.1.RELEASE.jar:/usr/local/avamar/lib/spring-asm-3.1.1.RELEASE.jar:/usr/local/avamar/lib/spring-beans-3.1.1.RELEASE.jar:/usr/local/avamar/lib/spring-context-3.1.1.RELEASE.jar:/usr/local/avamar/lib/spring-core-3.1.1.RELEASE.jar:/usr/local/avamar/lib/spring-expression-3.1.1.RELEASE.jar:/usr/local/avamar/lib/spring-amqp-1.2.0.RELEASE.jar:/usr/local/avamar/lib/spring-rabbit-1.2.0.RELEASE.jar:/usr/local/avamar/lib/spring-web-3.1.1.RELEASE.jar:/usr/local/avamar/lib/spring-jdbc-3.1.1.RELEASE.jar:/usr/local/avamar/lib/spring-tx-3.1.1.RELEASE.jar::/usr/local/avamar/lib/spring-security-config-3.1.0.RELEASE.jar:/usr/local/avamar/lib/spring-security-core-3.1.0.RELEASE.jar:/usr/local/avamar/lib/spring-security-crypto-3.1.0.RELEASE.jar:/usr/local/avamar/lib/spring-security-web-3.1.0.RELEASE.jar -Djava.rmi.server.hostname=clvdp01 -Djava.util.logging.config.file=/usr/local/avamar/lib/mcserver_logging.properties -Djava.security.egd=file:/dev/./urandom -Djava.io.tmpdir=/usr/local/avamar/var/mc/server_tmp -Djava.util.prefs.PreferencesFactory=com.avamar.mc.util.MCServerPreferencesFactory -Djavax.xml.parsers.DocumentBuilderFactory=org.apache.xerces.jaxp.DocumentBuilderFactoryImpl -Djavax.net.ssl.keyStore=lib/rmi_ssl_keystore -Djavax.net.ssl.trustStore=lib/rmi_ssl_keystore -Dfile.encoding=UTF-8 -Dlog4j.configuration=file:///usr/local/avamar/lib/log4j.properties com.avamar.mc.CLI --start

Is Anyone is able to help?

Best Regards,

Jacek

We want to use VDP as DR solution, to quickly bring up VMs in case the

VM somehow should get lost (e.g. accidentally deleted).

For some VMs that contain large data disks (several 2TB vdisks) we

wanted to only backup the system disks (data disks are backed up

conventionally to tape). These disks are configured as independent.

This is where we have trouble: while the selective backups succeed,

the VDP restore operations always try to set up every disk that is

configured in the VM, even if empty.

Consequently, with our sizy VMs this operation will quickly fail with

VDP: Restore Job Error

Insufficient disk space on datastore ...

VMWare KB 2110746 may be related.

Is there any workaround in VDP?

Kind regards, Tom

Hi all,

we have a fairly large vSphere 5.0 environment with 2x vCenter servers, each with ca. 2000 VMs using about 600 TB of FC-attached SAN storage.

The majority (~90%) of these VMs are backed up with Symantec NetBackup 7.5 via the vStorage API (doing VMware snapshots through vCenter).

While the environment was growing the backup environment also grew by adding more and more media servers to it. But we are still using only one vCenter server per location.

Since a while we are having trouble with a growing number of backup job failures: It looks like the backup servers' connections to vCenter sometimes fail or time out in the middle of a job.

And the vCenter servers (VMs with 8 vCPUs and 32 GB RAM) appear to be heavily loaded (esp. CPU, but also RAM).

My question here is: Does this happen, because we have reached an architectural limit of such kind of backup setup? Or in other words: Do we need to add additional vCenter servers and spread hosts and VMs among them?

We expect further growth of this environment and need to take a decision quickly. I cannot provide any details about how the backups are done right now. I'd just like to hear some opinions from people who run comparable environments and also use snapshot based backups. Should one vCenter server be able to handle backups of more than 2000 VMs? Is it worth trying to scale up the one vCenter server?

Thanks for any comments!

Andreas

Hi,

I'm using VDP to backup a Vcenter Linux Appliance amongst other VM, on a Vsphere 5.5 infrastructure.

Everything was just fine until the day the email notification indicated that de backup of my vmware appliance failed on every single backup (all the other vms are ok).

When I look further I realise that the backup is actually ok. I extracted the logs of failed clients from VDP web interface.

In the log there is only 1 warning.

Here is my log (warning in red...).

--------------------------

2014-04-14T19:00:15.456+4:00 avtar Info <5174>: - Reading /root/.avamar

2014-04-14T19:00:15.456+4:00 avtar Info <5174>: - Reading /usr/local/avamar/etc/usersettings.cfg

2014-04-14T19:00:15.456+4:00 avtar Info <5905>: Incremental flags: --flagfile=/usr/local/avamar/etc/usersettings.cfg --password=**************** --vardir=/usr/local/avamar/var --server=arsoidataprotect --id=root --bindir=/usr/local/avamar/bin --sequencenumber=105

2014-04-14T19:00:15.456+4:00 avtar Info <5946>: File system character encoding is UTF-8.

2014-04-14T19:00:15.456+4:00 avtar Info <5588>: - Creating process lock file "/usr/local/avamarclient/var-proxy-7/avtar.lck", pid=24942

2014-04-14T19:00:15.456+4:00 avtar Info <8940>: Starting back up at 2014-04-14 19:00:15 GST as "root" on "arsoidataprotect" (4 CPUs) [7.0.181-86]

2014-04-14T19:00:15.456+4:00 avtar Info <5731>: Not using include/exclude list

2014-04-14T19:00:15.456+4:00 avtar Info <8474>: - Log file path: Sauvegarde Vcenter Appliance-Sauvegarde Vcenter Appliance-1397487600017-31c6896d4f35d974baf96d89c25572c2985faa76-1016-vmimagel_avtar.log

2014-04-14T19:00:15.456+4:00 avtar Info <6555>: Initializing connection (Avamar Deduplication Engine v2.0.0)

2014-04-14T19:00:15.937+4:00 avtar Info <5552>: Connecting to Avamar Server (arsoidataprotect.sd.intranet.sante.gouv.fr)

2014-04-14T19:00:15.937+4:00 avtar Info <5554>: Connecting to one node in each datacenter

2014-04-14T19:00:15.940+4:00 avtar Info <5993>: - Connect: Connected to 10.153.6.54:29000, Priv=0, SSL Cipher=AES256-SHA

2014-04-14T19:00:15.944+4:00 avtar Info <5993>: - Datacenter 0 has 1 nodes: Connected to 10.153.6.54:29000, Priv=0, SSL Cipher=AES256-SHA

2014-04-14T19:00:15.947+4:00 avtar Info <5581>: Logging in on connection 0 with Session Ticket

2014-04-14T19:00:15.947+4:00 avtar Info <5582>: Avamar Server login successful

2014-04-14T19:00:15.947+4:00 avtar Info <5018>: - Session ID: 0

2014-04-14T19:00:15.949+4:00 avtar Info <10632>: Using Client-ID='31c6896d4f35d974baf96d89c25572c2985faa76'

2014-04-14T19:00:15.949+4:00 avtar Info <5550>: Successfully logged into Avamar Server [7.0.81-86] (Compression enabled)

2014-04-14T19:00:15.949+4:00 avtar Info <7562>: Back up of stdin on server "arsoidataprotect.sd.intranet.sante.gouv.fr" for /10.153.6.220/VirtualMachines/VMware_vCenter_Server_Appliance_UqELzCQ5Xb261oD3tCf27Q

2014-04-14T19:00:15.968+4:00 avtar Info <5586>: Loading cache files from /usr/local/avamarclient/var-proxy-7

2014-04-14T19:00:15.968+4:00 avtar Info <5769>: Filecache is disabled.

2014-04-14T19:00:15.968+4:00 avtar Info <8650>: Opening cache file /usr/local/avamarclient/var-proxy-7/p_cache.dat

2014-04-14T19:00:16.299+4:00 avtar Info <5573>: - Loaded cache file (100,663,840 bytes)

2014-04-14T19:00:16.664+4:00 avtar Info <6426>: Done loading cache files

2014-04-14T19:00:16.664+4:00 avtar Warning <18692>: Changed streamformat-out to none since raw format is only for DDR use

2014-04-14T19:00:16.664+4:00 avtar Info <6995>: Parsing 'pax' stream format from stdin

2014-04-14T19:00:16.667+4:00 avtar Info <8478>: Using backup #105 timestamp 2014-04-01 19:07:08 GST label "Sauvegarde vCenter Appliance-Sauvegarde vCenter " as base for incremental backup

2014-04-14T19:00:16.667+4:00 avtar Info <10818>: Backstream creating root backstreamdir object

2014-04-14T19:00:16.668+4:00 avtar Info <8648>: Directory elements will be sorted: No

2014-04-14T19:00:16.668+4:00 avtar Info <8649>: Sub-file change blocks enabled: Yes

2014-04-14T19:15:16.115+4:00 avtar Info <8688>: Status 2014-04-14 19:15:16, 22.87 GB (187.3 MB, 0.80% new) 388MB  19% CPU  24564711424 input bytes, (1 open files) VMFiles/1/virtdisk-flat.vmdk

2014-04-14T19:30:17.406+4:00 avtar Info <8688>: Status 2014-04-14 19:30:17, 48.90 GB (210.9 MB, 0.42% new) 392MB  16% CPU  52504117248 input bytes, (1 open files) VMFiles/2/virtdisk-flat.vmdk

2014-04-14T19:45:17.571+4:00 avtar Info <8688>: Status 2014-04-14 19:45:17, 97.26 GB (396.9 MB, 0.40% new) 392MB  30% CPU  104435810304 input bytes, (1 open files) VMFiles/2/virtdisk-flat.vmdk

2014-04-14T19:58:27.855+4:00 avtar Info <5163>: Backup complete, wrapping-up session with Server

2014-04-14T19:58:28.020+4:00 avtar Info <7238>: Stream complete: 134,217,800,704 bytes read from stream

2014-04-14T19:58:28.021+4:00 avtar Info <5156>: Backup #118 timestamp 2014-04-14 19:58:27, 0 files, 125.0 GB (0 files, 475.6 MB, 0.37% new)

2014-04-14T19:58:28.021+4:00 avtar Info <7539>: Label "Sauvegarde Vcenter Appliance-Sauvegarde Vcenter Appliance-1397487600017", scheduled to expire after 05/14/14 (2014-05-14 15:00:00 UTC), daily,weekly,monthly,yearly backup, backup type: synthetic_full

2014-04-14T19:58:28.021+4:00 avtar Info <6083>: Backed-up 125.0 GB in 58.21 minutes: 129 GB/hour (0 files/hour)

2014-04-14T19:58:28.026+4:00 avtar Info <5587>: Updating cache files in /usr/local/avamarclient/var-proxy-7

2014-04-14T19:58:28.032+4:00 avtar Info <5069>: - Writing cache file "/usr/local/avamarclient/var-proxy-7/p_cache.dat"

2014-04-14T19:58:28.382+4:00 avtar Info <5546>: Cache update complete /usr/local/avamarclient/var-proxy-7/p_cache.dat (96.0MB of 365MB max)

2014-04-14T19:58:28.391+4:00 avtar Info <7883>: Finished at 2014-04-14 19:58:28 GST, Elapsed time: 0000h:58m:12s

2014-04-14T19:58:28.399+4:00 avtar Info <8468>: Sending wrapup message to parent

2014-04-14T19:58:28.400+4:00 avtar Info <5314>: Command completed (1 warning, exit code 0: success)

-----------------------------------

Any ideas ?

Thanks.

Philippe

Environment:

• vCenter Server 5.1.0-1123961
• ESXi 5.1.0-1065491
• VDP 5.5.6.180

Followed the usual upgrade procedure:

1. VDP Appliance snapshotted
2. Mounted ISO containing 5.8.0 upgrade
3. Started upgrade from service web console

The problem: logging in on vSphere Webclient produces an "Error #1009" as follows...

Found this KB about "wrong URLs informed on vApp properties", but VDP is not an vApp and there is no such configuration... how can this be fixed without upgrade vCenter Server to 5.5u2?

Update #1: now even rolling back the snapshot to VDP 5.5.6, the vCenter Server still shows the Error #1009... and is also showing the wrong version information on it. Any help we'll appreciate.

Regards,

FABrizzolla

Finally, I decided to start my own thread about this.

Here is the post I added to an other one:

Hi,

I installed VDP for the first time yesterday at 1:00pm. The appliance is sitting on a host in a vCenter that's in another network.

Host has 4 nics, one for corporate network, another one for a factory isolated network, vMotion and another isolated network.

The appliance is connected via NFS to a NAS that's in the same switch over 1GbE links.

Took a backup of a vm (that's sitting on another ESX cluster: same vCenter) yesterday at night and all went well. Took 45 minutes to backup it up the first time.

But since the first minute of VDP installation, I'm looking at my network monitoring system and it shows me bursts of data every 5-10 minutes sending in/out between the nas and esx... Actually vdp appliance sending out and nas receiving in.

It is almost everytime around 257 155 kbit/s every 5-10 minutes in the graph. Nothing else is monitoring (like vmware sensors or wmi of any kind. just snmp traffic from the switch port and a ping to the esx host)

No other ESX shows that kind of traffic. If I look at the nas graph, same thing which leads me in the direction that the case is between these two.

I shuted down the application so I can really see if there is an issue with it or related to nas device directly...

A thing I also noticed is traffic is coming out of vmnic0 for last night backup at 10pm which is fine but immediately dropped after the 45 minutes of backup when done. The strange thing is that the excessive traffic comes out of vmnic2 which is hooked up to the private network and has nothing to do with VDP. But as soon as I close the VDP appliance vm, graphs drop and no problem at all.

So the final question is; where the hell does that traffic comes from and why is it going on the vmnic2 while the vm has nothing to do in that port group!

Actually I think i'll go hook up the nas in the same switch as the vmnic0 as i'm thinking about it.. I guess the traffic is coming from vmnic0 but the nas is hooked up in the same network as vmnic2 so it has to jump accross chips inside the esx host. But I wonder why it keeps polling 200Mbps of data every 5-10 minutes for a duration of 2 minutes though maybe someone could shed some light on that.

That was the reason why I saw traffic on both interfaces. Now, NAS is hooked up in the same switch on the corporate network as the ESX host which the appliance is sitting on.

The vdp vm has 4 datastores:

100GB OS is on the SAN

3x 1TB data on the nas over nfs

Host ESXi 5.5.0 build 2456374

vCenter 5.5.0 build 2442329

NAS is a Lenovo Iomega StorCenter ix4-300d

(I enabled jumbo frames all along the path.. no changes)

What is utilizing that much of bandwidth from the appliance? Is this normal? It's continiously transfering stuff and I can't figure out what that is!

Are these fragments from last night backup? It was not even 80GB total of backup .. shouldn't be that long? Backup said complete and dedup to 32GB from 80...

Please if you could be of any help that would be appreciated.

Thanks.

I've installed Exchange 2013 SP1 (current patch level).  I'm tying to install the Exchange Plug-in per vmware-data-protection-administration-guide-60.pdf, age 169.  VMware VDP Exchange Backup User Configuration Tool start and tells me "Current user is not a Domain Administrator" (see screen shot).  I'm logged into the machine using a Domain Administrator credentials, not the local machine.  The Exchange sever, Storage group, Mailbox store dropdowns are empty, apparently from an authority problem.

The KB Creating a VMware vSphere Data Protection (VDP) backup user account for exchange plugin with VDP Advanced (2073168) talks about creating a VDPBackupUser account manually.  Before I trot down this road, I thought I'd see if there is something obvious or basic that I have missed.

I have Data Recovery running for just over a week and the Integrity check has completed OK each day until now. Now it seems to hang at 33%. The current task name is 'Recatalog' and the Status is 'Checking integrity of deduplication store... Progress is 33%

How can I get more information as to what is happening. I have tried restarting the Data Recovery appliance. The Destination is a 250GB SAN vmfs store which shows 96GB free. The log just indicates 'Executing Integrity Check'. There are no errors on the restore points of the backups.

I am configuring Windows 2008 R2 64bit for BackupExec 2010 but when I run Diskpart Automount disable and automount scrub. After I rebooted the server 2008 still asked me do I want to initilize the presented VMFS partitions?

I have checked and automount is disabled? and I ran scrub again? I tried this on another machine and the same happened?

Is there something else I should be running?

Hi,

I am using VDR and getting this error for one of VMs. Failed to create snapshot for vmname, error -3960 ( cannot quiesce virtual machine)

Taking the sanpshot manually with quiesce enabled works just fine.

Any ideas?

Hello,

I installed vDP in lab environment to test the product.  Everything was working well for few days.  Since Saturday, I am facing with this error

“the most common cause for this error is that the time on the vdp appliance and your sso server are not in sync”

I checked esxi5.1 host (inside vcenter) to make sure the time are in sync with web client.    I am using NTP (pool.ntp.org) under configuration on esxi 5.1 and NTP client is enabled and running. 

How can, I resolve this error.  vDP appliance disconnect and when, I try to reconnect faced with above error.  Appreciate your help

Ram

The initial issues I've had with VDP forced me to start fresh and make a completely new guest.  I deployed an ova for the .5TB VM.  I installed it to an NFS datastore.

It all installed fine and is working.  I can connect.  I have not done a thing.  I didn't run an integrity check, or set up  backup or anything.  My graphs for bandwidth usage to the NFS datastore shows there is constant traffic going there even though it is in its blackout period.

I had a similar experience the first time I installed it.  It would use 100Mb of transfer on the switch to the datastore for doing NOTHING.   Now I used a small older datastore its only using about 40Mbit but still using traffic.

Is this intentional?  I didn't see anything about this in the documentation.

The tcpdump shows that VDP is contacting the datastore.  The datastore when I tcpdump there it shows coming from the HOST ip (not the vdp ip)  which is curious.

Anyone explain to me why this would be needing to touch the datastore when nothing is needed to be done?

Good day everyone

We were doing VCB backups with EMC Legato Networker using vSphere 4.0 and following the upgrade to 5.0 we are trying to VADP backups working.

Nearly there, but Networker can't connect to vSphere to pull the snapshot over - it's trying to get to https:/vSphereServer:443/VMServer/VMServer.vmx but cannot.

If I open up Internet Explorer 6 and put in https:/vSphereServer:443/ then I get "Page Cannot Be Displayed".

I have installed the VMWare certificate in the Trusted Root directory, but I realise that until I can browse to the https folder without errors then my backups won't work.

Any suggestions gratefully received.

My setup:

vSphere 5.0.0, 469512

4 VM Instacnes, all being backed up using Veeam 6.1

Using Veeam 6.1 Backup and Replication

Monthly Full with incrementals.

My backup jobs are failing on ONE of the 4 instances.  A cruical one at that.

Veeam support indicated my snapshots are the cause which is not a Veeam support issue but rather VMWare's support issue.  When I go into my vSphere Client, I see no snapshots in the snapshot manager.  However, I saw this error in my events:

Warning message on Email. This virtual
machine has 255 or more redo logs in a
single branch of its snapshot tree. The
maximum supported limit has been
reached, creating new snapshots will not
be allowed. To create new snapshots,
please delete old snapshots or consolidate
the redo logs.
warning

Then I went into the folder in the datastore and saw all the Email-000001.vmdk and Email-000001-ctk.vmdk through Email-000255.vmdk and Email-000255-ctk.vmdk files.  I then noticed the hard drive files the Hard Disk 1, 2 and 3 are using Email-000255.vmdk respectively.

How can I resolve?  I read around Google I two options:

1. consolidate but I don't have enough space nor time for as I understand it could take days.

2. Create a new VM and copy the disks over, but which ones?

3. Use vCenter Converter to convert it to another instance, although it keeps erroring out at Checking Free Space at Destination after 30 minutes.

Can't I just delete all the Email-000001.vmdk and Email-000001-ctk.vmdk files?

I was wondering if this is possible or if anyone has tried this.

ESXi 5.0 with update 1

DR Server 2.0.1

The idea I have is to Use DR Server on the ESXi host it will be backing up.

In other words, DR Server is a VM within the ESXi host.  So is VCenterServer by the way.

DR Server will be backing up all the VM's on the ESXi host it lies on.

It Will backup to a separate HD attached to the ESXi host, a different datastore than the VM's are on.

So far I have all this setup and running.  No problems backing up or restoring things.  It works well.

My question is ... What happens if I loose the datastore where all the VM's are stored and I need to restore the entire box.

My idea is this ... Since the Datastore the DR Server was writing to is separate, can I just rebuild the ESXi host then add DRServer  (and vcenterserver) and then add the Datastore from the original DR Server HD?  Will the catalogue rebuild and show all the original Servers so I can select them and restore them ?

-Thanks for any responses ...

-Richard

Hi to everyone

I need some help to regain access to my VDP appliance.

Environment:

vCenter server 5.1 listening to port 444

ESXi hosts 5.0 build 623860

VDP appliance 5.1.1

After first installation the backup job runs with no issues, but I cannot access file level restore web client with error "Login failed. HostSession is NULL"

I tried both with domain administrator account (used during appliance registration) and with SSO admin account (SYSTEM-DOMAIN\admin).

After a reboot I cannot access the VDP appliance at all from vsphere web client. If I try to login I am presented with the following pop-up:

"The VDP appliance is not responding. Please try your request again.

Would you like to be directed to the VDP configuration screen to troubleshoot the issue?"

Then I tried to re-register the appliance, but nothing changes.

Accessing the appliance through ssh, I sniffed some packets and noticed that I see traffic to the vcenter server on port 443 instead of 444 as expected (please note that before the reboot traffic was correctly sent to port 444):

root@dataprotection:~/#: tcpdump src or dst vcenter-ip and not port 22 and not port 8543

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode

listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes

10:11:26.453654 IP dataprotection.33703 > vcenter.https: S 2574417830:2574417830(0) win 5840 <mss 1460,sackOK,timestamp 95863 0,nop,wscale 8>

10:11:26.453834 IP vcenter.https > dataprotection.33703: S 4065981307:4065981307(0) ack 2574417831 win 8192 <mss 1460,nop,wscale 8,sackOK,timestamp 20223503 95863>

10:11:26.453849 IP dataprotection.33703 > vcenter.https: . ack 1 win 23 <nop,nop,timestamp 95863 20223503>

10:11:26.457113 IP dataprotection.33703 > vcenter.https: P 1:152(151) ack 1 win 23 <nop,nop,timestamp 95864 20223503>

I see the following errors in /space/avamar/var/vdr/server_logs/vdr-server.log:

Then followed these instructions:

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2044826

but when I tried to restart mcs service I got the following error:

root@dataprotection:~/#: dpnctl start mcs

Identity added: /home/dpn/.ssh/dpnid (/home/dpn/.ssh/dpnid)

dpnctl: INFO: Starting MCS...

dpnctl: INFO: To monitor progress, run in another window: tail -f /tmp/dpnctl-mcs-start-output-21179

dpnctl: ERROR: error return from "[ -r /etc/profile ] && . /etc/profile ; /usr/local/avamar/bin/mcserver.sh --start" - exit status 1

And the content of file vcenterinfo.cfg was yet correct:

vcenter-hostname=**************************

vcenter-port=444

vcenter-username=MYDOMAIN\Administrator

vcenter-password={AES}ZDAM9DqvJolBBEdCE5ha5g==

vcenter-sso-hostname=*************************

vcenter-sso-port=7444

Many thanks