Hi All,

I just got problem in my environment. Here it is:

Before, the VM was backup normally during last 1 year. But, last week, the backup was failed. Here is failed notification:

Processing SERVER-A (1xx.1xx) Error: DiskLib error: [4].Success Failed to retrieve next FILE_PUT message. File path: [[datastore_name] SERVER-A/SERVER-A.vmx]. File pointer: [0]. File size: [3338].

My Environment Info:

VM Os               : Windows Server 2003

VMware Esxi      : 5.5

Vcenter Version  : 6.0

Backup App        : Veeam B&R

Additional Information : backup job for other VM's run normally

Why this error happen? What to do to fix this issue?

any suggestion will be appreciated.

Best Regards,

Azlan Syah

Dear good afternoon

I am about to implement VDPA and my doubt is that during the installation never request the disk space that I want to assign to this service. At the time of selecting the data store where I want to install it I inform the available space. If I formatted all the datastore I lose all the virtual machines installed.

How is it the procedure?

First of all thank you very much for your time.

Greetings.

Hello dear community!

I'm currently facing an odd problem.

After upgrading to 6.1.5 at some point I've noticed that all my VDP appliances refusing to backup VMs in suspended state.

It's not even getting to snapshot creation phase.

VDP appliance log says: Error: An unexpected error occurred with error code "31107"

The session log (mccli activity get-log --id=xxxxxxx) says only: "Vm in Suspended State"

Although there are no problems with all other VMs.

I would appreciate any idease about what could be the reason for this problem.

Thanks!

In a vSphere 5.5 environment based on 4 ESXi 5.5 servers the vCenter is configured as a Windows Server 2008 R2 VM.

The vCenter VM got corrupted so I restored it from a 2 days old Veeam backup.

After restoring the VM the (restored) vCenter looks unable to communicate with the ESXi servers, mainly 2 of the 4 ESXi servers.

So I can move VMs between two of the 4 server but not to the other 2.

I restarted several times the vCenter with no success.

I cannot power off the ESXi nodes because there are production VMs, and I cannot move them for the above reasons...

How can I force the (restored) vCenter sybchronize with all the ESXi servers?

Regards

marius

I have recently setup a small new virtual environment.  One of the key features is a vDP backup appliance running on a host at physically separate location, with backups being performed over the network.  This allows for backups that are protected from disasters like fire.

I am trying to backup 3 VMs from one host to this backup appliance, using a single backup job.  Two of them are able to backup fine, but one VM is always erroring out with this error:

VDP: Backup job (Main Backup) failed to backup client Main Server.  Execution error: E10007:Miscellaneous Error.

I haven't been able to find much more of value, except poking around some text logs in the vDP appliance I saw some SSL communications errors in the nfcssl protocol.

I should also add that the VM in question is a Windows Server 2008 R2 standard guest OS.  I have set disk.EnableUUID to false, and I'm using vshadow.exe in a script file to prep the VSS for backup during the snapshot process.  However, one of the other VMs that is successfully backing up is the exact same server OS and configuration, and it is working fine.

Both hosts are Dell servers running ESXi 5.1.

Also, this VM is also the largest of the 3 VMs, with a 1 TB partition with about 100GB utilized.  At one point it had an RDM disk attached, but after reading that can cause problems I have detached it and still get the error.

Thanks in advance for any help!

Brant

VDP appliance oldest checkpoint as validated not latest checkpoint are validated. Any process to check specific checkpoint manually?

Thanks in advance.

Sorry for trouble again, not sure if this is correct category, moving this discussion from vCenter to vSphere.

hope get some help from you.

Hi, All

There is one question regarding VMware snapshot technology - CoW or RoW, confusing me for a long time, hope someone can help clarify this.

As we know that the difference between CoW and RoW is the location the changed data is written after a snapshot, like the snapshot space or original data area, for CoW, the changed data will be written to original data area instead of snapshot space after original data to be changed is copied to snapshot space, and for RoW, the changed data will be written to the space stores.

When take a snapshot for VMware, it creates some files, like delta disk files, .vmsd file, for delta disk files, it's a snapshot file and will store the changed data since last snapshot. So from this perspective, the snapshot technology VMware uses should be RoW, right?

However, from VMware guide or KB, we know that it uses CoW technology, as follows:

Understanding virtual machine snapshots in VMware ESXi and ESX (1015180) | VMware KB

The child disk

The child disk, which is created with a snapshot, is a sparse disk. Sparse disks employ the copy-on-write (COW) mechanism, in which the virtual disk contains no data in places, until copied there by a write. This optimization saves storage space. The grain is the unit of measure in which the sparse disk uses the copy-on-write mechanism. Each grain is a block of sectors containing virtual disk data. The default size is 128 sectors or 64 KB.

In addition, can anyone help explain the following snapshot chain regarding how it forms?

thanks

walker

Hi everyone,

I've tried everything I could for the past 2 days and I am still geting this...

I have reinstalled it 4 times so far and still same thing!

I CAN ping from within VDP shell the esxi hosts, vcenter by their FQDN, Hostname, IP and vice-versa

Hi all,

I just deployed VDP Appliance 6.1 for testing (download OVA file from https://my.vmware.com/group/vmware/details?productId=491&downloadGroup=VDP613#errorCheckDiv)

on my ESXi 5.5 lab. My question is, do i need to purchase the license for this Appliance?

Refer to https://communities.vmware.com/community/vmtn/vsphere/backup-recovery page, on VDP vs VDPA Comparison menu, there's no pricing for VDP. Thanks.

Is there a way to see what the status of each running job in VDP is? I know you can see percent complete of the overall job but is there a way to see which machine is being backed up, which have already been backed up and which proxy it is currently using?

I have 6 jobs that back up about 100 virtual machines and the jobs just seem to sit at the same percentage for a long time. They also seem to take an incredibly long time to run.

due to some reason , I have a abc-00002-delta.vmdk and no other files ( no VMX . no abc-0002-flat.vmdk)

is there any way to mount this vmdk in windows to recover files ？

or any way to mount this vmdk in vsphere ？

I tried to create a new VM and attach the vmdk , there is a error messages

"cannot open the disk '*.vmdk' or one of the snapshot disk is depends on"

We have deployed the OVA file and started the box. THe boxes gives a IP and advises us to connect to: https://<IP address>:8543/vdp-configure

All we get is a display page:

This web page is not available

Going to the IP without the port on the end displays:

Service unavailable!

The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.

If you think this is a server error, please contact the webmaster.

Error 503

Have tried redeploying the VM to see if anything happens and have also turned off our proxy settings.

any ideas?

Hi

We have a VMWare setup with 2 Nodes as below.

1. Physical Machine - Windows Server 2008 With vCenter 5.1

2. 2 Physical Hosts Running ESXi 5.1.0 HP Custom

We would like to upgrade the setup to 6.0 version with the conditions.

A. System is live & running 6 Virtual Machine, a minimum downtime is needed.

B. What should be the sequence for upgrade, first vCenter upgrade or ESX

C. Could we move all VMs to one ESX & upgrade the setup with 1 vCenter & 1 Node, then move all VMs to upgraded setup & then upgrade the rest & add to the cluster.

Kindly guide.

Thanks a lot!

I'm having trouble with my 1TB vdp appliance, the appliance configure status webpage is showing all services are running but when I login to the appliance and run the command  dpnctl status is shows gsan as degraded.

• I can't get backups to execute manually and they didn't run to schedule overnight.
• The appliance is showing disk usage around 36% but no backups are listed in the restore tab in vSphere web client.
• A manual integrity check completed successfully yesterday

I tried stopping/starting the gsan using dpnctl stop gsan, dpnctl start gsan but it returned to degraded fairly soon afterwards

I tried changing hte backup window so its not in maintenance mode to see if that would let me execute manual vm backups but that didn't work

I've also increased the RAM on the appliance from the default 4GB to 6GB

Can anyone tell me if gsan degraded is user fixable or if not will the appliance correct it itself and if so how long (minutes, a few hours, 0.5 days, more)

Hi All,

I have a vSphere 5.1 environment.  I installed VDP Advanced, went through the configuration successfully, but there's no VDP option on my web client.  The registration with vCenter said it was successful...so I'm not sure what I missed.

Any thoughts?

Thanks,

Lauren

Hi,

For few days VDP has not been able to complete integrity check launched from

web client so I tried running it manually:

1. Stop all VDP services (dpnctl stop)

2. Unmount the mount point (umount /data0X)

3. Run xfs_check (xfs_check /dev/sdxy)

4. Mount data0X (mount -a)

5. start services (dpnctl start all)

6. stop maintenance services (dpnctl stop maint)

7. create checkpoint (avmaint checkpoint --ava)

8. Run full hfscheck (avmaint hfscheck --full --ava)

8b. check hfscheck (avmaint hfscheckstatus --ava)

9. Start maint (dpnctl start maint)

Ref: https://communities.vmware.com/message/2271165

But every time I end up with error:

Last hfscheck failed with result MSG_ERR_HFSCHECKERRORS : started Tue Jan 14 16:02:19 2014 ended Tue Jan 14 20:42:22 2014, nodes queried/replied/total(1/1/1)

VDP version is 5.5.5.180

Any ideas what to try next?

Clean install is also possible, but I'd like to know the root cause for this and avoid it in the future.

Thanks!

Hey guys,

With a fresh VDP 6.1.3 install on top of vSphere 6.5, most backup jobs hang at 92%.  This is not the issue documented in the release notes, and the backups almost never complete. I DID NOT have this issue with VDP 6.1 under vSphere 6.0.

I have tried backing up photon OS machines, centos 6.5 machines, centos 6.6+ machines, and centos 7 machines.  So far the only ones to successfully complete are some of those old centos 6.5 ones.  I'm not sure if the O/S version is a red herring here, or if it's related to some other pattern I just can't identify at the time.

I have tried backing up images with the quiesce option both on and off, and it doesn't seem to make a difference in the result.  The backups hang regardless and are unsuccessful.

I should mention the storage was created along with the appliance and is on VSAN, if that makes any difference, but - this absolutely all worked with VDP 6.1 and vSphere 6.0.

I looked in a failed job's vmimage log in /usr/local/avamarclient/var/ and noticed this at the end:

2017-01-17T04:02:15.693+04:00 avvcbimage Info <9679>: Virtual disk connection to '[vsanDatastore-Neo-Cloud] 50c54958-6741-9944-301e-001ec93c076e/photonos-2cpu-4gb-is-vcpsc1.vmdk' has been closed
2017-01-17T04:02:15.693+04:00 avvcbimage Info <19718>: Found Disk from Device Key: '2001'.
2017-01-17T04:02:15.693+04:00 avvcbimage Info <9674>: virtual disk [vsanDatastore-Neo-Cloud] 50c54958-6741-9944-301e-001ec93c076e/photonos-2cpu-4gb-is-vcpsc1_1.vmdk will be connected readonly
2017-01-17T04:02:15.693+04:00 avvcbimage Info <16041>: VDDK:VixDiskLib: VixDiskLib_OpenEx: Open a disk.

2017-01-17T04:02:15.693+04:00 avvcbimage Info <16041>: VDDK:VixDiskLibVim: VixDiskLibVim_GetNfcTicket: Get NFC ticket for [vsanDatastore-Neo-Cloud] 50c54958-6741-9944-301e-001ec93c076e/photonos-2cpu-4gb-is-vcpsc1_1.vmdk.

2017-01-17T04:02:16.008+04:00 avvcbimage Info <16041>: VDDK:VixDiskLibVim: Request RandomAccessRO diskKey = 2001, readOnly = 1, openSnapshot = 0.

2017-01-17T04:02:16.094+04:00 avvcbimage Info <16041>: VDDK:VixDiskLibVim: VixDiskLibVim_FreeNfcTicket: Free NFC ticket.

2017-01-17T04:02:16.101+04:00 avvcbimage FATAL <5889>: Fatal signal 11 in pid 15065
2017-01-17T04:02:16.103+04:00 [VcbImageBackupAssistThread]  FATAL ERROR: <0001> uapp::handlefatal: Fatal signal 11
2017-01-17T04:02:16.103+04:00 [VcbImageBackupAssistThread]  | 00000000006b93b1
2017-01-17T04:02:16.103+04:00 [VcbImageBackupAssistThread]  | 00000000006ba127
2017-01-17T04:02:16.103+04:00 [VcbImageBackupAssistThread]  | 00000000006ba44b
2017-01-17T04:02:16.103+04:00 [VcbImageBackupAssistThread]  | 00000000006ba53e
2017-01-17T04:02:16.103+04:00 [VcbImageBackupAssistThread]  | 0000000000628810
2017-01-17T04:02:16.103+04:00 [VcbImageBackupAssistThread]  | 00007f2e56c90850
2017-01-17T04:02:16.103+04:00 [VcbImageBackupAssistThread]  | 00007f2e5050193f
2017-01-17T04:02:16.103+04:00 [VcbImageBackupAssistThread]  | 00007f2e5051c277
2017-01-17T04:02:16.103+04:00 [VcbImageBackupAssistThread]  | 00007f2e5051c75e
2017-01-17T04:02:16.103+04:00 [VcbImageBackupAssistThread]  | 00007f2e5052b142
2017-01-17T04:02:16.103+04:00 [VcbImageBackupAssistThread]  | 00007f2e5052b678
2017-01-17T04:02:16.103+04:00 [VcbImageBackupAssistThread]  | 00007f2e56ecab73
2017-01-17T04:02:16.103+04:00 [VcbImageBackupAssistThread]  | 00007f2e56ecb123
2017-01-17T04:02:16.103+04:00 [VcbImageBackupAssistThread]  | 00000000004284bc
2017-01-17T04:02:16.103+04:00 [VcbImageBackupAssistThread]  | 00000000004291f9
2017-01-17T04:02:16.103+04:00 [VcbImageBackupAssistThread]  | 00000000004993c4
2017-01-17T04:02:16.103+04:00 [VcbImageBackupAssistThread]  | 000000000070ac4c
2017-01-17T04:02:16.103+04:00 [VcbImageBackupAssistThread]  | 00007f2e56c88806
2017-01-17T04:02:16.103+04:00 [VcbImageBackupAssistThread]  | 00007f2e558cd9bd
2017-01-17T04:02:16.103+04:00 [VcbImageBackupAssistThread]  ERROR: <0001> uapp::handlefatal: aborting program pid=15065, sig=11
2017-01-17T04:02:16.103+04:00 avvcbimage FATAL <5890>: handlefatal: Aborting program with code 176, pid=15065, sig=11
2017-01-17T04:02:16.103+04:00 avvcbimage Info <16041>: VDDK:VixDiskLib: VixDiskLib_Exit: Unmatched Init calls so far: 1.

2017-01-17T12:00:07.999+04:00 avvcbimage Info <40660>: vcbimage_progress::terminate

the log ends there.

This looks serious to me, but it's hard to figure out where to go from here.

Does anyone have any advice as to how to proceed?

Thanks for any assistan

Long story short - a folder was deleted from the datastore which contained the flat and descriptor file for one drive on a server containing very important data. This was also the time when we found out the backups were corrupt. We have the *000001-ctk.vmdk, *000001-delta.vmdk, and 000001.vmdk file for this drive. Is it possible to recover the main descriptor and flat file (or the entire deleted directory) from this?

Any help is greatly appreciated

Hello,

After a Storage VMotion of the VDP appliance, it became unresponsive. I'm not able to manage it through vCenter nor its web interface. I guess it's something related to its internal proxy.

When I try to enable it, it fails. I've already rebooted the VDP appliance and the VCSA. Also, I've checked via CLI the health of the VDP services and the backup data. Seems fine.

I've also tried to unregister VDP VM and register again. I need help

My environment:

ESXi 6.0 U2

VCSA 6.0 build 3634794

VDP 6.1

Here it comes some screenshots and logs of the issue:

/usr/local/avamar/var/vdr/server_logs/vdr-server.log

2016-06-15 11:48:12,020 ERROR [Timer_CustomValue]-vi.ViJavaAccess: doGetAveVirtualMachine(): Unable to obtain local AVE virtual machine from its IP address [10.12.73.253]
2016-06-15 11:48:12,021 ERROR [Timer_CustomValue]-vi.ViJavaAccess: getAveVirtualMachine(): Unable to obtain local AVE virtual machine. Might be an Avamar physical machine.
2016-06-15 11:48:12,021 ERROR [Timer_CustomValue]-vi.ViJavaAccess: Failure to determine local AVE instance-uuid
2016-06-15 11:48:12,061 ERROR [Timer_CustomValue]-schedule.CustomValueUpdateTimerTask: Could not update the custom values.
java.lang.NullPointerException        at java.lang.String.contains(Unknown Source)        at com.emc.vdp2.schedule.CustomValueUpdateTimerTask.checkAndUpdateAnnotation(CustomValueUpdateTimerTask.java:97)        at com.emc.vdp2.schedule.CustomValueUpdateTimerTask.run(CustomValueUpdateTimerTask.java:72)        at java.util.TimerThread.mainLoop(Unknown Source)        at java.util.TimerThread.run(Unknown Source)
2016-06-15 11:48:16,869 ERROR [Timer_ServiceHealthValidator]-vi.ViJavaAccess: doGetAveVirtualMachine(): Unable to obtain local AVE virtual machine from its IP address [10.12.73.253]
2016-06-15 11:48:16,869 ERROR [Timer_ServiceHealthValidator]-vi.ViJavaAccess: getAveVirtualMachine(): Unable to obtain local AVE virtual machine. Might be an Avamar physical machine.
2016-06-15 11:48:16,869 ERROR [Timer_ServiceHealthValidator]-schedule.AlarmCheckTimerTask: Failed to get aveVM. aveVM is null
2016-06-15 11:48:16,869 INFO  [Timer_ServiceHealthValidator]-schedule.ServiceHealthValidatorTimerTask: Starting Service Health Check on Wed Jun 15 11:48:16 BRT 2016
2016-06-15 11:48:16,878 INFO  [Timer_retireVM]-schedule.RetireVirtualMachineTask: Searching for virtual machine registrations to retire...
2016-06-15 11:48:16,879 ERROR [Timer_HFSValidator]-vi.ViJavaAccess: doGetAveVirtualMachine(): Unable to obtain local AVE virtual machine from its IP address [10.12.73.253]
2016-06-15 11:48:16,879 ERROR [Timer_HFSValidator]-vi.ViJavaAccess: getAveVirtualMachine(): Unable to obtain local AVE virtual machine. Might be an Avamar physical machine.
2016-06-15 11:48:16,880 ERROR [Timer_HFSValidator]-schedule.AlarmCheckTimerTask: Failed to get aveVM. aveVM is null
2016-06-15 11:48:17,136 INFO  [Timer_retireVM]-connection.Mcsdk10StubManager: ServiceInstanceMoref desc=Service Id: urn:uuid:08FCE6D79ECB43283E1466002109508 name=urn:uuid:08FCE6D79ECB43283E1466002109508 value=SERVICE
2016-06-15 11:48:17,186 INFO  [Timer_retireVM]-connection.McAccessManager: Creating new mcsdk stub handler for connection key: [1151234205, Service Id: urn:uuid:08FCE6D79ECB43283E1466002109508] on Thread: [Timer_retireVM]
2016-06-15 11:48:17,240 INFO  [Timer_HFSValidator]-schedule.HFSValidatorTimerTask: HFS ValidRangeHours 48
2016-06-15 11:48:17,240 INFO  [Timer_HFSValidator]-schedule.HFSValidatorTimerTask: Last validated checkpoint happended on Tue Jun 14 15:06:25 BRT 2016
2016-06-15 11:48:17,360 INFO  [Timer_ServiceHealthValidator]-schedule.ServiceHealthValidatorTimerTask: Internal proxy configured?true
2016-06-15 11:48:17,360 INFO  [Timer_ServiceHealthValidator]-services.BaseCLI: GsanCLI: Updating service state.
2016-06-15 11:48:18,421 INFO  [GsanCLI-update]-services.BaseCLI$4: GsanCLI-update: State update complete. New state: running
2016-06-15 11:48:18,421 INFO  [Timer_ServiceHealthValidator]-services.BaseCLI: McsCLI: Updating service state.
2016-06-15 11:48:18,441 INFO  [McsCLI-update]-service.AdapterUtils: MCS Web Services URL: https://vdp6.epl.gov.br:9443/services/mcService  MCUserId="MCUser"
2016-06-15 11:48:18,632 INFO  [McsCLI-update]-connection.Mcsdk10StubManager: ServiceInstanceMoref desc=Service Id: urn:uuid:08FCE6D79ECB43283E1466002111165 name=urn:uuid:08FCE6D79ECB43283E1466002111165 value=SERVICE
2016-06-15 11:48:18,650 INFO  [McsCLI-update]-connection.McAccessManager: Creating new mcsdk stub handler for connection key: [1151234205, Service Id: urn:uuid:08FCE6D79ECB43283E1466002111165] on Thread: [McsCLI-update]
2016-06-15 11:48:18,699 INFO  [McsCLI-update]-services.McsCLI: MC Connection with Key: [McConnectionKey URI: https://vdp6.epl.gov.br:9443/services/mcsdk10 Username: MCUser Locale: en_US mcDomain: / trustAllSSLCerts: true productName: VDP] isConnected: [true]

Dear Team,

We want to transfer one Virtual Machine image from one site to second site, without effecting Source VM. want to know following 2 methods will work ....

Method 1

1) Take clone of Source VM

2) Copy new cloned vm from vmfs datastore to external HDD

3) ship the external hdd to second site

4) Connect external hdd on vcenter converter server

5) restored on any ESX host

or

Method 2

1) Take image backup of Source VM

2) manually copy image backup folder from VCB server to external HDD

3) ship the external hdd to second site

4) Connect external hdd on vcenter converter server

5) restored on any ESX host

want to know which option will be the best or any other options are available to do the needful.

regards

Mr VMware