My IT Notes

Difference between processors and cores in VM configuration:

https://blogs.vmware.com/vsphere/2013/10/does-corespersocket-affect-performance.html

In short: better use CPU Sockets, not Cores.

 

Disable vmem file in vmware player and ESXi:

Vmware Player creates .vmem file by default (memory swap) in VM folder. I don’t like it. To disable it configure
mainMem.useNamedFile = “FALSE”
in VMs .vmx file.
This action will significantly speed up the virtual machine and reduce the amount of HDD space it occupies :-)
To disable swap in ESXi:
Edit virtual machine settings -> Resourses -> Memory -> Reserve all guest memory
This will consume a fixed amount of memory from the total hypervisor resource, but it will speed up the machine and the system as a whole and, again, will not take up extra space in the datastore.

 

Make ESXi 6.5 work with old client:

vi /bootbank/clients.xml
<ConfigRoot>
<clientConnection id="0000">
<authdPort>902</authdPort>
<version>6</version>
<exactVersion>6.0.0</exactVersion>
<patchVersion>1.0.0</patchVersion>
<apiVersion>dev</apiVersion>
<downloadUrl>https://0.0.0.0</downloadUrl>
<flexClientVersion>6.0.0</flexClientVersion>
</clientConnection>
</ConfigRoot>
vi /etc/rc.local.d/local.sh
rm /usr/lib/vmware/hostd/docroot/client/clients.xml
cp /bootbank/clients.xml /usr/lib/vmware/hostd/docroot/client/clients.xml
chmod 444 /usr/lib/vmware/hostd/docroot/client/clients.xml

Reboot server and check how it works

https://lib.ru/unixhelp/vmware.txt#14

 

Enter ESXi shell using keyboard:

How to get in shell right from server’s keyboard?
https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere/8-0/esxi-installation-and-setup-8-0/installing-and-setting-up-esxi-install/setting-up-esxi-install/enable-esxi-shell-and-ssh-access-with-the-direct-console-user-interface-install.html
– From the Direct Console User Interface, press F2 to access the System Customization menu.
– Select Troubleshooting Options and press Enter.
– Enable ESXi Shell
Use Alt+F1 to enter Direct Console and Alt+F2 to exit from it.

 

Check and fix VMFS datastore after replacing disk in RAID:

https://www.settlersoman.com/how-to-check-and-fix-vmfs-metadata-using-vsphere-on-disk-metadata-analyzer-voma/

Stop background copy on ESXi:

# /sbin/services.sh restart

https://billyfung2010.blogspot.ru/2012/11/how-to-stop-file-copy-on-vmware-esxi-50.html

Restart web console:

https://communities.vmware.com/t5/VMware-vCenter-Discussions/ESXI-Web-UI-not-working/m-p/490841/highlight/true#M6923
/etc/init.d/hostd restart
/etc/init.d/vpxa restart
/etc/init.d/rhttpproxy restart

 

VMDK manipulations:

Rename VMDK:
vmkfstools -E OldName.vmdk NewName.vmdk

https://kb.vmware.com/s/article/1002491

Cloning or Converting a Virtual Disk or RDM:
-i|--clonevirtualdisk oldName newName
-d|--diskformat [thin|zeroedthick|eagerzeroedthick|rdm:device|rdmp:device|2gbsparse]
-W|--objecttype [file|vsan|vvol]
--policyFile fileName
-N|--avoidnativeclone

https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.storage.doc/GUID-01D3CF47-A84A-4988-8103-A0487D6441AA.html

Move VMDK:
vmkfstools -i "/vmfs/volumes/datastorename/vm/source.vmdk" -d thin "/vmfs/volumes/datastorename/vm/target.vmdk"

 

Check HDD performance via console:

cd /vmfs/volumes/[datastore]
time dd if=/dev/zero of=tempfile bs=8k count=1000000

 

Copy/move VM from one datastore to another:

cp -R /vmfs/volume/olddatastore/myvm /vmfs/volume/newdatastore/myvm
mv /vmfs/volume/olddatastore/myvm /vmfs/volume/newdatastore/myvm
https://forums.servethehome.com/index.php?threads/esxi-moving-vms-to-another-datastore.17508/post-168403

 

Copy VM from one ESXi host to another:

scp -p -r /vmfs/volumes/storagename/vmname/  root@remote.host.ip:/vmfs/volumes/storagename/vmname/

-r means recursive
-p preserves modification times, access times, and modes from the original file.

Enable SSH on the host you are copy VM from.
Enable SSH Client in destination host’s firewall.

https://www.vhersey.com/2013/05/copy-files-between-esxi-hosts-using-scp/

Here is another options:

var.1:

scp -v -c aes128-ctr -r /vmfs/volumes/datastore1/VMNAME* root@0.0.0.0:/vmfs/volumes/datastore1/VMNAME

var. 2:

Use OVFTOOL

syntax: ovftool.exe -ds= vi:/// vi://

Example: ovftool.exe -ds=DS-1 vi://10.1.1.10/SRV-WSUS vi://10.1.1.15

https://community.spiceworks.com/how_to/124377-copy-vmware-guest-from-one-esxi-host-to-another

https://www.virtuallyghetto.com/2012/06/how-to-copy-vms-directly-between-esxi.html

 

VMware ESXi cannot find the virtual disk

vmkload_mod multiextent

 

Менеджмент памяти vmware

 

Backup and restore ESXi config:

https://kb.vmware.com/s/article/2042141
Backup:
vim-cmd hostsvc/firmware/sync_config
vim-cmd hostsvc/firmware/backup_config

Restore:
!When restoring configuration data, the build number of the host must match the build number of the host on backup file and UUID (can be obtained using the command “esxcfg-info -u”) of the host should match the UUID of the host on backup file.
Use numeric 1 as force option to override the UUID mismatch. For example, vim-cmd hostsvc/firmware/restore_config 1 /tmp/configBundle.tgz
vim-cmd hostsvc/maintenance_mode_enter
vim-cmd hostsvc/firmware/restore_config /tmp/configBundle.tgz
You can see the .tgz using ls /scratch/downloads/

 

VMware ESXi 7.0 и неподдерживаемое оборудование

https://vmind.ru/2020/07/30/vmware-esxi-7-0-i-nepodderzhivaemoe-oborudovanie/

Restore Management network ESXi:

If happen :-) to remove Management Network and even in ESXi DCUI setting became unactive, grayed, than:

• esxcfg-vswitch -a vSwitch1
With this command you can create a new vSwitch named: “vSwitch1”

• esxcfg-vswitch -A “Management Network” vSwitch1
now you can assign a portgroup “Management Network” to the new vSwitch1

• esxcfg-vswitch -L vmnic0 vSwitch1
this step adds pNIC “vmnic0” to the vSwitch1

• esxcfg-vmknic -a  -i 160.xxx.xxx.xxx -n 255.xxx.xxx.xxx “Management Network”
this command will set the IP address/Subnet

https://www.running-system.com/total-loss-of-esxi-management-network-settings-rebuild-vswitch-portgroup-and-ip-settings-with-command-line/

Few useful commands:
– list existing vSwitches:
esxcfg-vswitch -l
– list existing vmks:
esxcli network ip interface list
– lis existing NICs:
esxcfg-nics l
– bind vSwitch to NIC:
esxcfg-vswitch -L vmnic0 vSwitch1
– unbind NIC:
esxcfg-vswitch -U vmnic0 vSwitch1
– remove vmKernel:
esxcli network ip tinterface name –interface-name vmkX

 

WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!

rm .ssh/known_hosts

 

Link aggregation (NIC teaming):

https://www.k-max.name/vmware/razbiraemsya-s-lacp-i-nic-teaming-v-vmware/

 

LSI storcli commands in ESXi:

http://pyatilistnik.org/storcli-komandyi-upravleniya-raid-kontrollerom-lsi-v-vmware-esxi-5-5/
/opt/lsi/storcli/storcli show

 

Add hardware dish to VM directly:

Via link:
vmkfstools -z /vmfs/devices/disks/diskname /vmfs/volumes/datastorename/vmfolder/vmname.vmdk

 

Записки на манжетах Максима Мошкова про VMware:

http://lib.ru/unixhelp/vmware.txt

 

Размер системной партиции OSDATA в ESXi 7.0:

https://williamlam.com/2020/05/changing-the-default-size-of-the-esx-osdata-volume-in-esxi-7-0.html

Не создается Datastore. Call “HostDatastoreSystem.QueryVmfsDatastoreCreateOptions” for object “ha-datastoresystem” on ESXi failed:

https://itbru.ru/index.php/2018/11/22/esxi-queryvmfsdatastorecreateoptions-for-object/
Go to Storage -> Devices
Click on the disk you need and you will see its path like “/vmfs/devices/disks/naa.600508b1001c86bd09bb2eb56ca8249b”

In console do this (all information on this disk will be erased!):
partedUtil mklabel /dev/disks/naa.600508b1001c86bd09bb2eb56ca8249b msdos

Now you can create the datastore.

Failed to deploy OVF package: The task was canceled by a user.

If OVA:
– unpack the OVA file with the virtual machine using Winrar or 7zip;
– change the extension of the .mf file (for example, to mf-)
– find the CD-ROM: vmware.cdrom.iso in the .ovf file
– change iso to atapi
– export the unpacked contents by selecting the edited .ovf file

https://communities.vmware.com/thread/431021

https://tm-experimental.blogspot.com/2013/11/failed-to-deploy-ovf-package-task-was.html

Cannot connect ‘path:0/1/0/5 version:2’ to this virtual machine. The device was not found.:

https://communities.vmware.com/t5/ESXi-Discussions/USB-quot-device-not-found-error-quot-when-adding-to-VM-ESXi-6-5/td-p/1405618
Add usb.generic.allowCCID = "TRUE" to the VM’s vmx file.

Protect ESXi server against encryption by Babuk-like ransomware:

https://kb.vmware.com/s/article/76372
https://www.bleepingcomputer.com/news/security/massive-esxiargs-ransomware-attack-targets-vmware-esxi-servers-worldwide/
https://www.gkhan.in/new-esxiargs-ransomware/
esxcli system slp stats get
/etc/init.d/slpd stop
esxcli network firewall ruleset set -r CIMSLP -e 0
chkconfig slpd off
chkconfig --list | grep slpd

E-token failed to add to VM with error “Failed to reconfigure virtual machine VMNAME. Cannot connect ‘vid: pid: path:0/1/0’ to this virtual machine. The device was not found”:

https://communities.vmware.com/t5/ESXi-Discussions/USB-eToken-ESXi-6-5-not-working/td-p/464775

1. Power of the VM
2. Append the following line to your VM configuration (.vmx)
usb.generic.allowCCID = "TRUE"

Free ESXi 8.0 – How to Download and get License Keys:

https://www.virten.net/2022/11/free-esxi-8-0-how-to-download-and-get-license-keys/

Mouse cursor misplaced in vSphere Client and in VMRC:

In Windows 11 when you apply custom Scale in Display settings there may be some misplacement of cursor in VMRC ans vSphere VM’s console.
It can be fixed by configuring DPI settings in Compatibiality options for the corresponding application:
VpxClient.exe -> Compatiability -> Change High DPI settings -> Use this setting to fix scaling… -> Use DPI that’s set for my main display when -> I open this program
and
VpxClient.exe -> Compatiability -> Change High DPI settings -> High DPI scaling override -> Application

ESXi Network Speed Test Step-By-Step using iperf:

https://whyiblog.com/esxi-network-speed-test-step-by-step/
https://www.virtualizationhowto.com/2019/02/run-basic-network-speed-bandwidth-throughput-test-between-esxi-hosts/

iSCSI Storage ESXi 7:

https://www.server-world.info/en/note?os=ESXi_7&p=storage&f=4

Useful ESXi 7 info:

https://www.server-world.info/en/note?os=ESXi_7

Configure NUMA node affinity:

https://docs.pexip.com/server_design/vmware_numa_affinity.htm
https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.resmgmt.doc/GUID-7E0C6311-5B27-408E-8F51-E4F1FC997283.html#GUID-7E0C6311-5B27-408E-8F51-E4F1FC997283
cpuid.coresPerSocket = 1
numa.vcpu.preferHT = TRUE
numa.nodeAffinity = SOCKET-NUMBER
Don’t forget to set number of cores:
numa.autosize.vcpu.maxPerVirtualNode = NUMBER-OF-CORES

Check NUMA node RAM usage by VM:

esxtop
m (to select Memory stats)
f (to configure the view options)
uncheck everything except D and G (name and NUMA)
return (exit to the esxtop)
V (configure view of running VMs only)

GST_NDX – NUMA node number where VM’s memory allocated in

Online Update Standalone ESXi Host:

https://knowledge.broadcom.com/external/article?legacyId=2143832

List of ESXi releases and updates: current versions and unsupported.
https://esxi-patches.v-front.de/ESXi-8.0.0.html

In most cases you’ll just need to:
esxcli network firewall ruleset set -e true -r httpClient
esxcli software profile update -p ESXi-7.0U3s-24585291-standard \
-d https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml
esxcli network firewall ruleset set -e false -r httpClient

But sometimes you may have .depot file:
https://docs.macstadium.com/docs/update-standalone-esxi-host-via-online-bundle
https://www.nakivo.com/blog/instal-vib-files-on-esxi/
Put host in Maintenance Mode:
vim-cmd hostsvc/maintenance_mode_enter
Check software profile in depot file:
esxcli software sources profile list --depot=file:///vmfs/volumes/DATASTORENAME/VMware_ESXi_RELEASE_depot.zip
Check update with needed depot file using corresponding profile:
esxcli software profile update --depot=file:///vmfs/volumes/DATASTORENAME/vib/VMware_ESXi_RELEASE_depot.zip --profile=ESXi-PROFILENAME --dry-run
Sometimes --no-hardware-warning shold be applied.
If everything is ok, then update the host:
Update host with needed depot file using corresponding profile:
esxcli software profile update --depot=file:///vmfs/volumes/DATASTORENAME/vib/VMware_ESXi_RELEASE_depot.zip --profile=ESXi-PROFILENAME
Reboot
Exit maintenance mode:
vim-cmd hostsvc/maintenance_mode_exit

Error while updating ESXi: VIB QLC_bootbank_qedrntv_ requires qedentv_ver = , but the requirement cannot be satisfied within the ImageProfile

When you’re updating ESXi from .zip package, using esxcli software vib install -d /vmfs/volumes/PATH-TO-UPDATE.zip command, system can throw an error about the ImageProfile.
esxcli software sources profile list -d /vmfs/volumes/PATH-TO-UPDATE.zip
this will show a list of profiles. Choose one that fits your needs:
esxcli software profile update -p ESXi-XXX-standard -d /vmfs/volumes/PATH-TO-UPDATE.zip
wait until install ends

Read SSD SMART TBW ESXi CLI:

https://www.virten.net/2016/05/determine-tbw-from-ssds-with-s-m-a-r-t-values-in-esxi-smartctl/
Download: http://www.virten.net/files/smartctl-6.6-4321.x86_64.vib
Install:
esxcli software acceptance set --level=CommunitySupported
esxcli software vib install -v /vmfs/volumes/NAME/smartctl-6.6-4321.x86_64.vib
list available devices in ESXi shell:
ls -l /dev/disks/
list all info about selected device:
/opt/smartmontools/smartctl -d [Device Type] --all /dev/disks/[DISK]
for SATA disks use –sat
/opt/smartmontools/smartctl -d sat –all /dev/disks/[DISK]
list only SMART attributes about selected device:
/opt/smartmontools/smartctl -d sat --attributes /dev/disks/[DISK]
list only needed attribute information:
/opt/smartmontools/smartctl -d sat --attributes /dev/disks/[DISK] | grep Temperature_Celsius

Interestingly that on some SSD’s only 241 Total_LBAs_Written attribute is available:
241 Total_LBAs_Written 0x0032 100 100 000 Old_age Always - 122520
This number does not mean anything that is connected to a real written Bytes, Gigabytes ot Terabytes.
I’ve tried to use the formula from https://www.virten.net/2016/05/determine-tbw-from-ssds-with-s-m-a-r-t-values-in-esxi-smartctl/ but in my case number was 5,720462650060654e-5
I did an experiment:
dd if=/dev/zero of=/vmfs/volumes/STORAGENAME/test/test bs=1M count=1024
and looked how “241 Total_LBAs_Written” changed from 122357 to 122520. Using simple caclulations: 163 * X = 1024 we have X = near 6.3. It seems that every megabyte written increases the number in “241 Total_LBAs_Written” by 6.3
Now it is possible to count the TBW of that SSD: (122520 * 6.3)/1024 = 753,7 Gigabytes

SMART attributes by vendor: https://www.smartmontools.org/wiki/Attributes_VendorDocs

Change VMRC locale:

https://tklaassens.wordpress.com/2019/05/16/change-gui-language-of-vmware-remote-console/
Edit “%APPDATA%\VMware\preferences.ini”:
Add the following line to it and save the file:
pref.locale = "en_US"

Export VM from ESXi 7 & 8 to OVA in Windows:

Export VM:
"C:\Program Files\VMware\VMware OVF Tool\ovftool.exe" vi://USERNAME@XXX.XXX.XXX.XXX/VMNAME LOCALDISK:\vmname.ova
Make sure thet you use lowercase letters when specifying the export format (.ova). If you use uppercase, this strange ovftool.exe will create .ovf with a bunch of .vmdk and other files and you will need to use this code to merge them all in one .ova file:
"C:\Program Files\VMware\VMware OVF Tool\ovftool.exe" LOCALDISK:\vmname.ovf LOCALDISK:\vmname.ova
Mybe Vmware just want us to wear our SSDs faster? I don’t know any other reason why it is working like this. In previous versions (before 7) of ESXi everything was good: you could just command to export as OVA and get everything done.

Add existing iSCSI datastore to ESXi:

https://www.reddit.com/r/esxi/comments/15ganm8/mount_existing_vmfs_datastore_to_esxi_host/
https://web.archive.org/web/20220928065153/https://communities.vmware.com/t5/ESXi-Discussions/Need-to-add-an-existing-local-datastore-from-before-a-re-install/td-p/490383
Sometimes when you’re adding an existing iSCSI datastore to the newly created ESXi, you can stop at the step when you can see the iSCSI devise added but no datastore apeeared and you can’t add any by clicking the “New Datastore” button.
Thy looking if ESXi sees needed datastore:
esxcli storage vmfs snapshot list
if yes then:
esxcfg-volume -M "VMFS UUID"
P.S. Special thanks to the assholes and assheads in Broadcom that broke all community and KB Vmware links.

The certificate assigned to this host is not valid yet. You should install a valid certificate:

https://www.virten.net/2023/07/regenerate-standalone-esxi-host-certificate/

/sbin/generate-certificates
/etc/init.d/hostd restart

“Note: This method should not be used when the ESXi host is already added to a vCenter Server. In that case, the certificate should be renewed using Right-Click ESXi Host in Inventory > Certificates > Renew Certificate”

Make Removable HDD/SSD Fixed:

Add to the .vmx file devices.hotplug = false

SHA Digest of the file doesn’t match manifest:

https://www.youtube.com/watch?app=desktop&v=NNpF-5sLC0g&t=66s
In most cases it says that something wrong with import-export process, or vmdk files are broken due to some reasons.
But sometimes removing (or renaming) the .mf file helps import the VM (or part/s of it).

VMware ESXi Patch Tracker:

How to install:
esxcli software vib install -d “/vmfs/volumes/Datastore/DirectoryName/PatchName.zip”
https://kb.vmware.com/s/article/2008939

How to uninstall:
https://pyatilistnik.org/kak-udalit-vib-paket-v-esxi-5-5-6-0/
esxcli software vib list
esxcli software vib remove -n packsge-name

Move ESXi syslog to different location:

https://www.nakivo.com/blog/change-vmware-esxi-logs-location/
When ESXi is installed on SD card or any other non-reliable storage device, it’s better to move logs, generated by ESXi to another location.
Steps to do:
– Check current configuration:
esxcli system syslog config get
– Get real path of all datastores:
esxcli storage vmfs extent list
– Configure new log location:
esxcli system syslog config set --logdir=/vmfs/volumes/VOLUME/syslog
– Reload log service:
esxcli system syslog reload
No reboot is required after this configuration.

Tons of hacks in .vmx for ESXi and Workstation:

https://gist.icoder.workers.dev/ddan9/daa3c1d3bce0eb879cd711d144712206

Check VMwate Tools Version:

https://www.dell.com/support/kbdoc/en-us/000227059/dell-vxrail-how-to-update-the-vmware-tools-version-in-the-esxi-host
esxcli software component get | grep VMware-VM-Tools -B 1 -A 14

ESXi install on server with CPU having P and E cores (PSOD with Fatal CPU mismatch):

https://williamlam.com/2023/01/video-of-esxi-install-workaround-for-fatal-cpu-mismatch-on-feature-for-intel-12th-gen-cpus-and-newer.html
During the bootup of the ESXi installer press SHIFT+O and add cpuUniformityHardCheckPanic=FALSE to existing kernel boot options
After finishing the ESXi install do not reboot, press ALT+F1 and login to the ESXi shell using root and blank password. Edit /vmfs/volumes/BOOTBANK1/boot.cfg:
add cpuUniformityHardCheckPanic=FALSE to the kernelopt= line.
Exit shell by pressing ALT+F2 and then reboot.
To permanently configure the ESXi kernel boot setting, in case of updates/upgrades in the future:
esxcli system settings kernel set -s cpuUniformityHardCheckPanic -v FALSE

Additional info about P and C cores, how to detect and use them:
https://williamlam.com/2024/01/experimenting-with-esxi-cpu-affinity-and-intel-hybrid-cpu-cores.html

Limit Size of ESX-OSData while installing ESXi 7 & 8:

https://williamlam.com/2020/05/changing-the-default-size-of-the-esx-osdata-volume-in-esxi-7-0.html
Press Shift+O at the beginning of the ESXi install process and input this command:
autoPartitionOSDataSize=32768

Install ESXi on unsupported hardwere (CPU):

https://vinfrastructure.it/2025/01/installing-esxi-8-with-an-unsupported-cpu/
Press Shift+O at the beginning of the ESXi install process and input this command:
allowLegacyCPU=true