Hello,

I have a problem with VDP backups.

Yesterday I restarted my ESXi servers and my SAN. We are using 2x Dell PowerEdge R805 for hosts and a Dell PowerVault MD3000i SAN.

Because we are changing to a PowerVault MD3600i, I unplugged some iSCSI cables (while the server were shut down). Now each host is connected with only 1 iSCSI cable to the SAN.

After starting the servers, everything was running fine, expect the VDP backups.

VDP displays the following error message:

E30931 Failed to initiate a backup or restore for a virtual machine because no proxy was found to service the virtual machine

Sadly I didn't found something about this error, so I hope someone here can help me!

I expanded the space on the VDP 6 appliance from 2TB to 4TB by expanding the files (from 3 to 6).  But it is still showing over 80% full and growing after several days.  It should be around 40% now.  How can I force it to rebalance the backups between the new files or at least to acknowledge that it can use the new space?  It does say the capacity is 4.1TB, but it says the deduplicated size is 3.3TB which is totally bogus.

According to status.dpn, the garbage collection just ran.  It took nearly 2 hours and "recovered 12.09 GB".  So garbage collection didn't rebalance the files.

hfscheck ran yesterday successfully, so that didn't rebalance.

It has been thru a half dozen 12 hour maintenance windows. no luck.

All services are running, the only alarm is the VDP: [002] The VDP appliance is nearly full.

So how to I get it to rebalance?

I've been using VDP for the past few years (on the latest version). My backup and replication jobs are running fine, and I perform backup restores about 4 to 8 times a year.

The web access has always been slow. It takes about 15-20 minutes to load the backup jobs tab. And every time I make a change to a backup job the screen has to load again and it takes another 15-20 minutes.  What's the deal?  Is this normal for you guys?

All my backups are currently going to a Data Domain. I have about 12 jobs that backs up 28 VMs.  Can anyone help me figure out why it takes that long to load up the backup jobs list?  This also happens in the restore and report tab as well.

Hi,

In my environment we use Symantec Netbackup 7.1. We backup VM's using our vCenter server to a Netbackup 5000 device & the backups are then copied out to our Disaster Recovery site over a 100MB link.

In our Disaster Recovery site we have 3 ESXi 4.1 hosts & no vCenter server. Is it possible to restore a VM directly to a host instead of going through vCenter?  

Thanks. 

I'm looking at using Veeam for a DR solution. It looks like I am not the first to experience this issue http://communities.vmware.com/message/1900559#1900559

Problem: I am unable to successfully backup a VM via the management server using Veeam. I am however able to backup via a specific host in the cluster.

The Veeam support guys rightly said it's a VMWare issue highlighting that we can backup directl from a host but not from the vcenter server...

Thanks for any input!

Brendan

*********************

After running a few tests, we are now sucessfully backing up via Veeam.

Our management server was not presented any of the LUNs when it was originally configured. So I created a test LUN and presented to hosts and the management server, created a couple of vms and the backup went through fine.

I was also able to install a virtualized vCenter 5 server and all jobs processed fine.

It makes sense that the management server needs to be ‘LUN aware’

I just noticed that the (4) VDP appliances I am testing all stopped sending the Email Summary Report after the Daylight Savings time change on Sunday. the last report was generated Sunday morning at 12:00am. Now all the appliances are reporting this:

VDP: Send email summary report error. The email summary report timer is set to the incorrect hour. It does not match the vdrdb value.

The time this error is generated is 11:00:00pm which is exactly one hour from when it should be running.

I think this is a Daylight Savings time issue but dont know how to address it.

Any help would be greatly appreciated.

RickH

Hi,

I installed VDP appliance (5.1.1). Registered to the vcenter, setup several backup jobs. The appliance worked for 2 months quite reliably, but suddenly  something happened and I cannot start any job. Neither scheduled  not manual. The Web GUI reports that the job was succesfully scheduled, but the backup does not start. The problem lasts 2 weeks.

I have restarted VDP several times, executed integrity checks, but it does not help.

All appliance services are running (checked by health check by VDP web interface):

VDP status in Vcenter Web Client is "Normal" :

I have noticed, that in /space/avamar/var/mc/server_log/mcserver.log.0 there are lot of  repeating messages :

03/07-02:17:59.00667 com.avamar.mc.wo.JobScheduler._gotVmWork
WARNING: Backup job skipped because server is read-only
03/07-02:17:59.00667 com.avamar.mc.wo.JobScheduler.gotReadOnlyWork
FINE: bug 4261 --- checking for readonlywork
03/07-02:17:59.00667 com.avamar.mc.cr.bcConnection.sendWOReply

The time of the message is evidently in backup window. It seems, that the appliance is internally in readonly mode for some reason, but i am not able find out why.

     Any ideas ?

          Thanks

               Jiri

I have been getting this error on several VMs but have not been able to figure out why.  I found the following in the log file.  Can someone tell me why VDP cannot create the directories?  How can I fix?

2016-02-09T22:39:45.917+06:00 avvcbimage Info <11986>: Changed block tracking is engaged for this VM

2016-02-09T22:39:45.917+06:00 avvcbimage Info <11988>: A reference to a valid prior backup is not available so this will be a full level zero backup.

2016-02-09T22:39:45.959+06:00 avvcbimage Info <19549>: metadata tmp dir: /usr/local/avamarclient/var/vmware/2/temp

2016-02-09T22:39:45.959+06:00 avvcbimage Info <40640>: create tmpDir: /usr, failed (-1)

2016-02-09T22:39:45.959+06:00 avvcbimage Info <40640>: create tmpDir: /usr/local, failed (-1)

2016-02-09T22:39:45.959+06:00 avvcbimage Info <40640>: create tmpDir: /usr/local/avamarclient, failed (-1)

2016-02-09T22:39:45.959+06:00 avvcbimage Info <40640>: create tmpDir: /usr/local/avamarclient/var, failed (-1)

2016-02-09T22:39:45.959+06:00 avvcbimage Info <40640>: create tmpDir: /usr/local/avamarclient/var/vmware, failed (-1)

2016-02-09T22:39:45.959+06:00 avvcbimage Info <40640>: create tmpDir: /usr/local/avamarclient/var/vmware/2, failed (-1)

2016-02-09T22:39:45.960+06:00 avvcbimage Info <40640>: create tmpDir: /usr/local/avamarclient/var/vmware/2/temp, failed (-1)

2016-02-09T22:39:45.980+06:00 avvcbimage Info <40654>: isExitOK()=0

I'm using VDP 6.1.1

Hi All,
When we take snapshot through NetApp VSC plugin most of the VMs not backed up due to the below error. Is there any other method available? How can i fix this? Please suggest me on this.

Date-WARN - VMware Task "CreateSnapshot_Task" for entity "VMNAME" failed with the following error: An error occurred while quiescing the virtual machine. See the virtual machine's event log for details.
Date-WARN - Only a crash consistent backup will be taken for VM "VMNAME" since VMware snapshot create operation failed.

Hi everyone,

I've tried everything I could for the past 2 days and I am still geting this...

I have reinstalled it 4 times so far and still same thing!

I CAN ping from within VDP shell the esxi hosts, vcenter by their FQDN, Hostname, IP and vice-versa

Hello experts Veeam Backup & Replication

The essence of the problem. There are 2 Repository: 1-fast, 2-slow.

For a night on the repository 1 saved copies of virtual machines

(Buckup, Reversed Incremental, Restore points=3).

At 7:00 begins "Backup Copy Job", which makes a copy of a copy of a night on the repository 2.

(Restore points to keep = 3, Monthly backup = 3 in 1 day of the month). Default

creates 1 Full copy + 2 Increment and 3 Full copies of the last three

months. Is it possible by regular means, or a script to create 3 full copies of 3

last days and 3 Full copies of the last three months using Veeam Backup

& Replication 9.0.

Thanks in advance for your reply.

The environment is as follows:

A 2 node physical MS Cluster was made a single node MS Cluster and then P2V was performed to a VMware VM.   This was done because the belief that the DB2 and MQ Series based application was a MS Cluster and would need to continue as such.

So we now have a single node MS Cluster running under a single VM with VMDK disk under MS Cluster control.

Troubleshooting through the Windows Event Log:

A "Create virtual machine snapshot" was executed and completed successfully.

A VDP "snapshot" via proxy appliance was performed and "crashed".   The log shows a "VDP "Backup / Restore VM" (I am told it was just a backup, no restore) was executed and immediately followed with a MS Cluster "Error 1038" - loss of physical disk resources. 

Some 12 hours later a "Remove snapshot" was performed and exactly the time stamp that was done, the "Backup / Restore VM" showed Complete!

Questions:

1) Does VDP support a single node MS Cluster with vdsk?   (I see no indication of that, but client felt it was)

2) If yes to 1), any specific settings needed for it to work?

3) Any insights or comments on what could have been the reason why the VDP backup resulted in an immediate Err 1038 as described above?

Thanks in advance!

Environment:

• vCenter Server 5.1.0-1123961
• ESXi 5.1.0-1065491
• VDP 5.5.6.180

Followed the usual upgrade procedure:

1. VDP Appliance snapshotted
2. Mounted ISO containing 5.8.0 upgrade
3. Started upgrade from service web console

The problem: logging in on vSphere Webclient produces an "Error #1009" as follows...

Found this KB about "wrong URLs informed on vApp properties", but VDP is not an vApp and there is no such configuration... how can this be fixed without upgrade vCenter Server to 5.5u2?

Update #1: now even rolling back the snapshot to VDP 5.5.6, the vCenter Server still shows the Error #1009... and is also showing the wrong version information on it. Any help we'll appreciate.

Regards,

FABrizzolla

Hey All,

Experiencing an issue where my jobs for VMs running on local disk are failing due to what appears to be a timeout on the take snapshot action. So at first I thought it was a timing issue, so I looked at that. It takes about 1 minute for quiesed snapshots to actually happen.

I found the other articles that mention you should change the --subprocesstimeout=600 parameter to the /usr/local/avamarclient/var/avvcbimageAll.cmd file, so I did that with no luck. I also tried increasing the snapshot removal time, with no luck.

Still, after applying this and rebooting the appliance no dice, even though the snapshot is completing before the VDP task, it's still failing.

VMware KB: vSphere Data Protection backup jobs fail intermittently

I had another VM work just fine that was on a local SSD, but any disks that are otherwise located on slow local magnetic disk fail.

Looking at the logs, it would appear the most interesting entry is bolded and underlined below, which leads me to the following article, which says, your storage sucks, figure out why your snapshots are taking so long. Anyone have an idea how to increase the timeout when snapshots are being "TAKEN" as opposed to being removed? Next up will try removing queising from the config.

VMware KB: Backing up vCenter Server with vSphere Data Protection (VDP) fails with error: Soap fault due to Connecti…

2015-01-23T11:42:45.635+04:00 avvcbimage Info <19704>: DataStore Storage Info:ESX4_Local_ESXi_Installable capacity=1995012308992  free=1682109890560

2015-01-23T11:42:45.635+04:00 avvcbimage Info <19716>: DS Capacity=1995012308992  FreeSpace=1682109890560  / HD committed=57196542066    unCommitted=29917971642    unShared=57196542066

2015-01-23T11:42:46.584+04:00 avvcbimage Info <16001>: Found 2 disk(s), 0 snapshots, and 0 snapshot files, on the VMs datastore.

2015-01-23T11:42:46.584+04:00 avvcbimage Info <0000>: isExitOK()=0

2015-01-23T11:42:46.599+04:00 avvcbimage Info <19680>: vmAction runBackupScript: ()

2015-01-23T11:42:46.599+04:00 avvcbimage Info <19681>: vmAction runBackupScript: script is skipped because it is null

2015-01-23T11:42:46.600+04:00 avvcbimage Info <0000>: [IMG0009] Pre-snapshot script:  completed successfully

2015-01-23T11:42:46.600+04:00 avvcbimage Info <9692>: a VM snapshot has been requested

2015-01-23T11:42:46.600+04:00 avvcbimage Info <14627>: Creating snapshot 'VDP-1422031366f2a36877d42af6c1a25a994c67b393fa02286299', quieceFS=1

2015-01-23T11:42:46.635+04:00 avvcbimage Info <14631>: Snapshot 'VDP-1422031366f2a36877d42af6c1a25a994c67b393fa02286299' creation for VM '[ESX4_Local_ESXi_Installable] <VMNAME>/<VMNAME>.vmx' task still in progress, sleep for 2 sec

2015-01-23T11:42:48.662+04:00 avvcbimage Info <14631>: Snapshot 'VDP-1422031366f2a36877d42af6c1a25a994c67b393fa02286299' creation for VM '[ESX4_Local_ESXi_Installable] <VMNAME>/<VMNAME>.vmx' task still in progress, sleep for 2 sec

2015-01-23T11:42:50.689+04:00 avvcbimage Info <14631>: Snapshot 'VDP-1422031366f2a36877d42af6c1a25a994c67b393fa02286299' creation for VM '[ESX4_Local_ESXi_Installable] <VMNAME>/<VMNAME>.vmx' task still in progress, sleep for 2 sec

2015-01-23T11:43:12.796+04:00 avvcbimage Warning <16004>: Soap fault detected, Query problem, Msg:'SOAP 1.1 fault: SOAP-ENV:Client [no subcode]

"Name or service not known"

Detail: getaddrinfo failed in tcp_connect()

'

2015-01-23T11:43:12.796+04:00 avvcbimage Error <17773>: Snapshot 'VDP-1422031366f2a36877d42af6c1a25a994c67b393fa02286299' creation for VM '[ESX4_Local_ESXi_Installable] <VMNAME>/<VMNAME>.vmx' task failed to start

2015-01-23T11:43:12.796+04:00 avvcbimage Info <19680>: vmAction runBackupScript: ()

2015-01-23T11:43:12.796+04:00 avvcbimage Info <19681>: vmAction runBackupScript: script is skipped because it is null

2015-01-23T11:43:12.796+04:00 avvcbimage Info <0000>: [IMG0009] Post-snapshot script:  completed successfully

2015-01-23T11:43:12.796+04:00 avvcbimage FATAL <0000>: [IMG0003] The VMX '[ESX4_Local_ESXi_Installable] <VMNAME>/<VMNAME>.vmx' could not be snapshot.

2015-01-23T11:43:12.796+04:00 avvcbimage Info <9772>: Starting graceful (staged) termination, Create Snapshot failure. (wrap-up stage)

2015-01-23T11:43:12.796+04:00 avvcbimage Error <0000>: [IMG0009] createSnapshot: snapshot creation  or pre/post snapshot script failed

2015-01-23T11:43:12.796+04:00 avvcbimage Error <0000>: [IMG0009] createSnapshot: snapshot creation/pre-script/post-script failed

2015-01-23T11:43:12.796+04:00 avvcbimage Info <0000>: isExitOK()=202

2015-01-23T11:43:12.796+04:00 avvcbimage Info <40370>: snapshot created:false NOMC:false ChangeBlTrackingAvail:true UsingChBl:true, ExitOK:false, cancelled:false, fatal: true

2015-01-23T11:43:12.796+04:00 avvcbimage Info <0000>: vcbimage_progress::terminate

2015-01-23T11:43:12.796+04:00 avvcbimage Info <16041>: VDDK:VixDiskLib: VixDiskLib_EndAccess: Disk access completed.

Basically the article says, your storage sucks, deal with it and get better storage.

2015-01-23T11:43:12.796+04:00 avvcbimage Info <16041>: VDDK:VixDiskLib: VixDiskLib_Connect: Establish connection.

2015-01-23T11:43:12.796+04:00 avvcbimage Info <16041>: VDDK:VixDiskLibVim: VixDiskLibVim_AllowVMotion: Enable VMotion.

2015-01-23T11:43:12.798+04:00 avvcbimage Info <16038>: Final summary, cancelled/aborted 0, snapview 0, exitcode 202: plugin error 02

2015-01-23T11:43:14.907+04:00 avvcbimage Info <17819>: VixDiskLib vMotion reservation successfully released

--------------------------------------------------------------------------------------------------------

----- END avvcbimage log 2015-01-23 11:43:17 EST  (1 warning, 4 errors, 1 fatal error)

--------------------------------------------------------------------------------------------------------

Hallo VMware Community,

we are running VSA Backup of CommVault SIMPANA. We are using a VSA Proxy. To the Proxy the VMFS-5 LUNs are mapped read-only so that we can use SAN as transport mode for the VMs which are located on that datastores.

We have testet many things but always still getting the error message "Failure Reason:Could not read data from the virtual disk" and the Job failed. At the beginning we started with one Subclient for all the VMs but now we switched to one Subclient per VM so that the jobs are running sequentially step by step but the error still occurs.

Error Code (at SIMPANA Console):

Description: Unable to read the disk [] for virtual machine [xxx] on Host [xxx]. [Beim Verarbeiten der Anfrage ist mindestens ein Fehler aufgetreten., Failed to ReadCurrVmdkFile (W32.774)]

Source: xxx, Process: vsbkp    

8340  1b88  01/25 00:33:11 89467 stat- ID [readdisk], Bytes [26381910425], Time [133.847090] Sec(s), Average Speed [187.973832] MB/Sec

8340  1b88  01/25 00:33:11 89467 CVMDiskInfo::DecrementMaxNDBHandlesAllowed() - Max allowed NBD Open handles for backup are now [0]

8340  1b88  01/25 00:33:11 89467 CVMDiskInfo::IncrementMaxNDBHandlesAllowed() - Max allowed NBD Open handles for backup are now [1]

8340  1b88  01/25 00:33:11 89467 VSBkpWorker::BackupVMDisk() - Using default extent size

8340  1b88  01/25 00:33:11 89467 VSBkpWorker::BackupVMDisk() - Disabling CRC Processing as change Id [52 47 9b 90 ff f5 be 24-37 6c f9 43 a2 89 58 e3/57] is supplied

8340  1b88  01/25 00:33:11 89467 VSBkpChangeIdFile::ReadChangeID() - Not using a Change ID

8340  6     01/25 00:33:11 89467 ### GetService --- Reopening connection

8340  6     01/25 00:33:12 89467 ### CVIWrapper::_GetDiskInfoArr --- Updating disk label from [Festplatte 1] to [Hard disk 1]

8340  6     01/25 00:33:13 89467 ### CVIWrapper::_GetDiskInfoArr --- Updating disk label from [Festplatte 2] to [Hard disk 2]

8340  1b88  01/25 00:33:13 89467 There are 6 entries in the allocated extent table for xxx.vmdk

8340  1b88  01/25 00:33:13 89467 VSBkpWorker::BackupVMDisk() - Extent Size [262144] bytes [512] sectors [512] bytes per sector

8340  1b88  01/25 00:33:13 89467 VSBkpWorker::BackupVMDisk() - Sending File Header Path:[\501de18e-3138-67f9-2514-daffc1b51880\scsi0-1-xxx_1.vmdk] with size:[450971566080]

8340  1b88  01/25 00:33:13 89467 VSBkpWorker::AddVMMetadataFile() - Adding metadata file [D:\Program Files\CommVault\Simpana\iDataAgent\JobResults\CV_JobResults\iDataAgent\VirtualServerAgent\2\375\VolHdr(501de18e-3138-67f9-2514-daffc1b51880_scsi0-1-xxx_1.vmdk).imf] to list to be uploaded to index cache

8340  1b88  01/25 00:33:13 89467 VSBkpWorker::BackupVMDisk() - Backing up disk/volume [xxx_1.vmdk]

8340  1b88  01/25 00:33:13 89467 VSBkpWorker::BackupVMDisk() - Using Fixed Length Extent Names

8340  234c  01/25 00:33:13 89467 VSBkpWorker::ArchiveExtentProcessor() - Send volume or disk data to MA thread

8340  2134  01/25 00:34:35 89467 VSBkpCoordinator::OnIdle_Running() - Waiting for [1] VMs to be processed.  [xxx]

8340  2134  01/25 00:34:35 89467 VSBkpCoordinator::DumpStreamInfo() - StreamId = 1, RCID = 189844, ReservationId = 137886, MediaAgent = s190a0010, Agent = s190a0010  []

8340  1b88  01/25 00:34:50 89467 CVMDiskInfo::ReadDisk() - VixDiskLib_Read failed Err=3e80 One of the parameters supplied is invalid

8340  1b88  01/25 00:34:50 89467 VSBkpWorker::BackupVMDisk() - Failed to read from virtual disk [xxx_1.vmdk],  [0x80070306:{..\..\..\CommClient\VirtualServer\VMInfo\ArchiveVirtualDiskFile.cpp[CArchiveVirtualDiskFile::GetNextBlock(535)]} + {..\..\..\CommClient\VirtualServer\VMInfo\VmdkVcb4ExtentReader.cpp[CVmdkVcb4ExtentReader::ReadBlock(664)]/W32.774.(Beim Verarbeiten der Anfrage ist mindestens ein Fehler aufgetreten.)-Failed to ReadCurrVmdkFile}]

We are running the latest version of VMware ESXi and also the latest version of the vSphere Server.

Maybe someone of you is also using VSA backup of SIMPANA and can help us to resolve this issue.

Thanks a lot

Best Regards

Philipp

I want to ask that how to exclude the backup of a drive in vmware mirage server.

Its starting to look like another Chrome update is once again breaking the VDP management page. Trying to access the 8543/vdp-configure page on vSphere Data Protection 6.0.2.4 with Chrome 45 (beta build 45.0.2454.15) is returning an ERR_SSL_WEAK_SERVER_EPHEMERAL_DH_KEY and refusing to connect. Firefox 39 is also failing to connect with a similar error while Safari 8.0.7 works.

For the windows users I was able to connect with both Internet Explorer 11 (11.0.10240.16384) and Edge (20.10240.16384.0) on Windows 10.

Is anyone else having problems? I have not yet opened a support ticket as it's not a big enough problem for me to derail an ongoing project.

Hi every one,

I am doing some test with DP 6.0 and the SQL backup, I have been doing some backups but I could not restore any of the backups yet.

Every time is the same error no matter if I do some changes during the restore process.

I can see these lines at the events panel from the vsphere client :

Task: VDP: Restore Job

VDP Restore "VM name" started

VDP server execution error code: 10007 Job name ......

VDP: Restore completed

Could you please tell me where to check if there is another log where I can find more information about this error?

Also, I have done backups and restores for individual VMs and it works fine, until now I have done backups from SQL Application but I could not complete the restore correctly.

Thanks

Hi,

I've been following the Admin guide for vDP 6.0 and application server backups. There's one anomaly which is on page 179 at this step (step 3)

1 On the Data Type page, select Full Server and click Next.

2 On the Backup Sources page:

  a Click the arrow beside Microsoft SharePoint Servers(s) to expand the list.

  b Select the checkbox next to the SharePoint Server that you want to back up.

  c Click Next.

  a In the Farm Administrator Credentials section, type the administrator Login ID and Login

password;

  b Select the Enable multi‐stream backup option to allow multiple threads of execution during the

backup. Use the slider barto select the number of streams to use, and then select Database or Volume

in the Group by menu;

  c Click Next

Firstly, in the previous step, I see the four sharepoint servers I have installed the application client on and select them accordingly. On step 3 though, I don't get an option to select the single front end server to make the backup through. I just see the space to insert my credentials.

Setup:

1 x SP web servers (application/front-end client installed)

1 x SP web server (application/back-end client installed)

2 x SP SQL servers (application/back-end client installed)

When I run the job if I just put the 1 web server with front tend client on it succeeds, but if I add the other SP servers in at step 2, I get 3/4 failures stating that backups must be done through the fornt end server only. Is this natural behaviour in 6.1? As the admin guide is for 6.0. Just wondering if I'm missing something obvious here?

This has happened several times in the past. Each time requiring tech support from VMware.  I'd like to be able to do this myself.

A virtual machine has a lock file present in its directory.  The VM cannot be consolidated.  Can't take snapshots, and can't back it up with VDP.

VMname.vmx.lck is in the directory in the datastore.

When I go to the VDP appliance, I see that it has an extra disk attached to it.

It has a disk belonging to the VM that can't be consolidated.  This disk is located in the directory with the lockfile.

In the past, I've been able to remove these extra disks that get stuck to VDP (by editing VDP-- and removing the disk.  A VMware tech told me to try this approach).

But in this case,when I try to remove the disk from VDP I get the following error

"Cannot remove virtual disk from the vitual machine because it or one of its parent disks is part of a snapshot of a virtual machine.)

I can't just delete the disk lock either (at least not when browsing the datastore)-- access is denied.

I'm sure others have experienced this.

Any ideas?