Azure - AzCopy学习
使用 AzCopy 将本地数据迁移到云存储空间
-
azcopy login
- 创建存储账号
./azcopy login --tenant-id 40242385-c249-4746-95dc-4a0b64d49dc5- 这里的—tenant-id 在下面的地方查看：目录 ID；需要拥有Storage Blob Data Owner 的权限
- 账号下可能会有很多目录（tenant），所以在 login 的时候带上 —tenant-id
-
使用 azcopy copy 上传
azcopy copy 20230509_144654178_iOS.jpg https://eastorageaccountpg.blob.core.windows.net/private-pg修改文件名和 endpoint 即可。
如果使用 az 命令也可以打到效果
az storage blob upload -f 20230509_144654178_iOS.jpg -c blog-pg -n 20230509_144654178_iOS.jpg
AzCopy Sync 使用
1. azcopy sync，Client local Dir → EU：Blob/container/dir- Client未登录 Azure- blob 的 container 生成共享访问令牌，注意权限选择
Client 执行 azcopy sync 命令with SAS
root@ubuntu0:~# azcopy sync "./blog/" "https://eustoragesacc0.blob.core.windows.net/eucontainer?sp=racwdlep&st=2023-07-21T12:27:30Z&se=2023-07-21T20:27:30Z&spr=https&sv=2022-11-02&sr=c&sig=edSES09iSGU1xChuOAJnEKNrTJ1xbk0pR97aO%2FA0boE%3D"INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder supportJob e08f5117-f8b7-314d-5aa3-3f22b0fdf82b has startedLog file is located at: /root/.azcopy/e08f5117-f8b7-314d-5aa3-3f22b0fdf82b.log100.0 %, 2 Done, 0 Failed, 0 Pending, 2 Total, 2-sec Throughput (Mb/s): 0.6465Job e08f5117-f8b7-314d-5aa3-3f22b0fdf82b SummaryFiles Scanned at Source: 2Files Scanned at Destination: 0Elapsed Time (Minutes): 0.0667Number of Copy Transfers for Files: 2Number of Copy Transfers for Folder Properties: 0 Total Number Of Copy Transfers: 2Number of Copy Transfers Completed: 2Number of Copy Transfers Failed: 0Number of Deletions at Destination: 0Total Number of Bytes Transferred: 423810Total Number of Bytes Enumerated: 423810Final Job Status: Completed
查看 Blob 情况
-
EA：blob/container/dir → EU：blob/container/dir
-
ea 源生成 SAS，注意权限
-
Client 执行 azcopy sync 命令with SAS
root@ubuntu0:~# azcopy sync "https://eastorageacc0.blob.core.windows.net/eacontainer/sync/?sp=rle&st=2023-07-21T12:42:42Z&se=2023-07-21T20:42:42Z&spr=https&sv=2022-11-02&sr=c&sig=%2FZzdp1B5dpZzTGT3PlwhssFKnipWkgp%2BBpOIqo9t1mY%3D" "https://eustoragesacc0.blob.core.windows.net/eucontainer/sync/?sp=racwdlep&st=2023-07-21T12:27:30Z&se=2023-07-21T20:27:30Z&spr=https&sv=2022-11-02&sr=c&sig=edSES09iSGU1xChuOAJnEKNrTJ1xbk0pR97aO%2FA0boE%3D" INFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder supportJob 747202e5-1afc-b846-4388-9e2e4eee6c58 has started Log file is located at: /root/.azcopy/747202e5-1afc-b846-4388-9e2e4eee6c58.log100.0 %, 3 Done, 0 Failed, 0 Pending, 3 Total, 2-sec Throughput (Mb/s): 0 Job 747202e5-1afc-b846-4388-9e2e4eee6c58 Summary Files Scanned at Source: 3 Files Scanned at Destination: 0 Elapsed Time (Minutes): 0.1 Number of Copy Transfers for Files: 3 Number of Copy Transfers for Folder Properties: 0 Total Number Of Copy Transfers: 3 Number of Copy Transfers Completed: 3 Number of Copy Transfers Failed: 0 Number of Deletions at Destination: 0 Total Number of Bytes Transferred: 847620 Total Number of Bytes Enumerated: 847620 Final Job Status: Completed -
查看 Blob 情况
-
-
AzCopy copy 示例
azcopy copy "https://eascc20230728.blob.core.windows.net/local/1.txt?sp=racwli&st=2023-07-31T03:54:41Z&se=2023-08-31T11:54:41Z&spr=https&sv=2022-11-02&sr=c&sig=hRxFYgG3chofx1WUwlb0ji5wPuI%2BgA31DxUNZlPjAW4%3D" ./1.txt- 加上 token 要使用“”
- https://eascc20230728.blob.core.windows.net/local/1.txt 具体的 blob 还是目录要指定清楚
Blob bw Regions test
| 序号 | 说明 | 测试 1 | 测试 2 | 测试 3 | 平均值 |
|---|---|---|---|---|---|
| 1 | EA client → SA Container | 0.2334 | 0.2334 | ||
| 2 | SA Container → EU client | 1.0263 | 1.2675 | 1.0148 | 1.10286667 |
| 3 | SA Container → EU client @ peer created | 1.1209 | 1.2449 | 1.1483 | 1.17136667 |
| 4 | SA Container → EU Container | 4.1681 | 4.335 | 4.6351 | 4.3794 |
| 2 Hours Later | 1.4338 | 1.8673 | 1.65055 | ||
| 5 | EU Contaner → SA Container | 1.0003 | 0.9336 | 0.96695 | |
| 6 | EU client → EU Container | 0.767 | 0.7669 | 0.767 | 0.76696667 |
| 7 | EU Container → EU Client, VM disk iops limited | 0.8814 | 0.8814 | ||
| 8 | EA client →EU Container | 0.5335 | 0.7002 | 0.61685 | |
| 9 | EU container → EA container | 0.4001 | 0.4001 |
结论：
1. Client 跨Region 获取 Container 中文件，在 Blog 选择微软内网优先的情况下，Region间是否使用 Peer 对等对传输效率基本无影响；
2. Container 的文件直接跨 Region 传输效率与选择的 Region以及时机/方向 有关，一段时间保持稳定；
3. 同 Region 传输效率高；
原始数据
1. EA client → SA Containerroot@ubuntu0:/home/zyi# azcopy copy 1.zip https://sastorageac0.blob.core.windows.net/sacontainerINFO: Scanning...INFO: Autologin not specified.INFO: Authenticating to destination using Azure ADINFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder supportJob f631ebf2-8060-6b47-7164-58044ee0c43c has startedLog file is located at: /root/.azcopy/f631ebf2-8060-6b47-7164-58044ee0c43c.log100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total, Job f631ebf2-8060-6b47-7164-58044ee0c43c summaryElapsed Time (Minutes): 0.2334Number of File Transfers: 1Number of Folder Property Transfers: 0Number of Symlink Transfers: 0Total Number of Transfers: 1Number of File Transfers Completed: 1Number of Folder Transfers Completed: 0Number of File Transfers Failed: 0Number of Folder Transfers Failed: 0Number of File Transfers Skipped: 0Number of Folder Transfers Skipped: 0TotalBytesTransferred: 10519930170Final Job Status: Completed2. SA Container → EU clientroot@eu-ubuntu0:/home/zyi# azcopy copy https://sastorageac0.blob.core.windows.net/sacontainer/1.zip .INFO: Scanning...INFO: Autologin not specified.INFO: Authenticating to source using Azure ADINFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder supportJob bf9d0ff0-6066-2140-4cb9-8716e4272028 has startedLog file is located at: /root/.azcopy/bf9d0ff0-6066-2140-4cb9-8716e4272028.log0.0 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s):0.0 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2.1 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s):6.2 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s):11.0 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s)15.9 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s)20.3 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s)24.6 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s)29.1 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s)33.7 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s)38.4 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s)42.4 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s)46.9 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s)51.4 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s)55.3 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s)59.7 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s)64.4 %, 0 68.9 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s): 1710.624 (Disk may be l73.4 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s): 1845.0545 (77.3 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s): 1440.6049 82.5 %, 0 Done, 0 Failed, 1 Pending, 0 Skipped, 1 Total, 100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total, Job bf9d0ff0-6066-2140-4cb9-8716e4272028 summaryElapsed Time (Minutes): 1.0263Number of File Transfers: 1Number of Folder Property Transfers: 0Number of Symlink Transfers: 0Total Number of Transfers: 1Number of File Transfers Completed: 1Number of Folder Transfers Completed: 0Number of File Transfers Failed: 0Number of Folder Transfers Failed: 0Number of File Transfers Skipped: 0Number of Folder Transfers Skipped: 0TotalBytesTransferred: 10519930170Final Job Status: Completed#download 2root@eu-ubuntu0:/home/zyi# azcopy copy https://sastorageac0.blob.core.windows.net/sacontainer/1.zip .INFO: Scanning...INFO: Autologin not specified.INFO: Authenticating to source using Azure ADINFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder supportJob 732e37c9-9e99-364a-4a4e-fa7484fac927 has startedLog file is located at: /root/.azcopy/732e37c9-9e99-364a-4a4e-fa7484fac927.log100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total, Job 732e37c9-9e99-364a-4a4e-fa7484fac927 summaryElapsed Time (Minutes): 1.2675Number of File Transfers: 1Number of Folder Property Transfers: 0Number of Symlink Transfers: 0Total Number of Transfers: 1Number of File Transfers Completed: 1Number of Folder Transfers Completed: 0Number of File Transfers Failed: 0Number of Folder Transfers Failed: 0Number of File Transfers Skipped: 0Number of Folder Transfers Skipped: 0TotalBytesTransferred: 10519930170Final Job Status: Completed#download 3^[[Aroot@eu-ubuntu0:/home/zyi# azcopy copy https://sastorageac0.blob.core.windows.net/sacontainer/1.zip .INFO: Scanning...INFO: Autologin not specified.INFO: Authenticating to source using Azure ADINFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder supportJob 05c9210a-e503-c046-6f9b-853d6c9a3c4e has startedLog file is located at: /root/.azcopy/05c9210a-e503-c046-6f9b-853d6c9a3c4e.log100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total, Job 05c9210a-e503-c046-6f9b-853d6c9a3c4e summaryElapsed Time (Minutes): 1.0148Number of File Transfers: 1Number of Folder Property Transfers: 0Number of Symlink Transfers: 0Total Number of Transfers: 1Number of File Transfers Completed: 1Number of Folder Transfers Completed: 0Number of File Transfers Failed: 0Number of Folder Transfers Failed: 0Number of File Transfers Skipped: 0Number of Folder Transfers Skipped: 0TotalBytesTransferred: 10519930170Final Job Status: Completed```3. SA Container → EU client @ peer created```powershellroot@eu-ubuntu0:/home/zyi# azcopy copy https://sastorageac0.blob.core.windows.net/sacontainer/1.zip .INFO: Scanning...INFO: Autologin not specified.INFO: Authenticating to source using Azure ADINFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder supportJob b1d8da46-9358-6b4c-66d4-bb8cb8d53e1f has startedLog file is located at: /root/.azcopy/b1d8da46-9358-6b4c-66d4-bb8cb8d53e1f.log100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total, Job b1d8da46-9358-6b4c-66d4-bb8cb8d53e1f summaryElapsed Time (Minutes): 1.1209Number of File Transfers: 1Number of Folder Property Transfers: 0Number of Symlink Transfers: 0Total Number of Transfers: 1Number of File Transfers Completed: 1Number of Folder Transfers Completed: 0Number of File Transfers Failed: 0Number of Folder Transfers Failed: 0Number of File Transfers Skipped: 0Number of Folder Transfers Skipped: 0TotalBytesTransferred: 10519930170Final Job Status: Completed#2root@eu-ubuntu0:/home/zyi# azcopy copy https://sastorageac0.blob.core.windows.net/sacontainer/1.zip .INFO: Scanning...INFO: Autologin not specified.INFO: Authenticating to source using Azure ADINFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder supportJob b646e5c1-af07-6c41-7a0c-641ecfd5d1cd has startedLog file is located at: /root/.azcopy/b646e5c1-af07-6c41-7a0c-641ecfd5d1cd.log100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total, Job b646e5c1-af07-6c41-7a0c-641ecfd5d1cd summaryElapsed Time (Minutes): 1.2449Number of File Transfers: 1Number of Folder Property Transfers: 0Number of Symlink Transfers: 0Total Number of Transfers: 1Number of File Transfers Completed: 1Number of Folder Transfers Completed: 0Number of File Transfers Failed: 0Number of Folder Transfers Failed: 0Number of File Transfers Skipped: 0Number of Folder Transfers Skipped: 0TotalBytesTransferred: 10519930170Final Job Status: Completedroot@eu-ubuntu0:/home/zyi# azcopy copy https://sastorageac0.blob.core.windows.net/sacontainer/1.zip .INFO: Scanning...INFO: Autologin not specified.INFO: Authenticating to source using Azure ADINFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder supportJob 30f70c4e-fdd1-9a4b-6e89-c5e7406970de has startedLog file is located at: /root/.azcopy/30f70c4e-fdd1-9a4b-6e89-c5e7406970de.log100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total, Job 30f70c4e-fdd1-9a4b-6e89-c5e7406970de summaryElapsed Time (Minutes): 1.1483Number of File Transfers: 1Number of Folder Property Transfers: 0Number of Symlink Transfers: 0Total Number of Transfers: 1Number of File Transfers Completed: 1Number of Folder Transfers Completed: 0Number of File Transfers Failed: 0Number of Folder Transfers Failed: 0Number of File Transfers Skipped: 0Number of Folder Transfers Skipped: 0TotalBytesTransferred: 10519930170Final Job Status: Completed```1. SA Container → EU Container```powershellroot@eu-ubuntu0:/home/zyi# azcopy copy https://sastorageac0.blob.core.windows.net/sacontainer/1.zip https://eustorageac0.blob.core.windows.net/eucontainer/1.zip INFO: Scanning...INFO: Autologin not specified.INFO: Authenticating to destination using Azure ADINFO: Authenticating to source using Azure ADINFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder supportJob 9168f92f-2da4-6742-6862-8e5696b52a41 has startedLog file is located at: /root/.azcopy/9168f92f-2da4-6742-6862-8e5696b52a41.logINFO: Transfers could fail because AzCopy could not verify if the destination supports tiers.100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total, Job 9168f92f-2da4-6742-6862-8e5696b52a41 summaryElapsed Time (Minutes): 4.1681Number of File Transfers: 1Number of Folder Property Transfers: 0Number of Symlink Transfers: 0Total Number of Transfers: 1Number of File Transfers Completed: 1Number of Folder Transfers Completed: 0Number of File Transfers Failed: 0Number of Folder Transfers Failed: 0Number of File Transfers Skipped: 0Number of Folder Transfers Skipped: 0TotalBytesTransferred: 10519930170Final Job Status: Completedroot@eu-ubuntu0:/home/zyi# azcopy copy https://sastorageac0.blob.core.windows.net/sacontainer/1.zip https://eustorageac0.blob.core.windows.net/eucontainer/10.zip INFO: Scanning...INFO: Autologin not specified.INFO: Authenticating to destination using Azure ADINFO: Authenticating to source using Azure ADINFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder supportJob 7cca86e8-11b6-164d-7b33-887d9cf10183 has startedLog file is located at: /root/.azcopy/7cca86e8-11b6-164d-7b33-887d9cf10183.logINFO: Transfers could fail because AzCopy could not verify if the destination supports tiers.100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total, Job 7cca86e8-11b6-164d-7b33-887d9cf10183 summaryElapsed Time (Minutes): 4.335Number of File Transfers: 1Number of Folder Property Transfers: 0Number of Symlink Transfers: 0Total Number of Transfers: 1Number of File Transfers Completed: 1Number of Folder Transfers Completed: 0Number of File Transfers Failed: 0Number of Folder Transfers Failed: 0Number of File Transfers Skipped: 0Number of Folder Transfers Skipped: 0TotalBytesTransferred: 10519930170Final Job Status: Completed#3root@eu-ubuntu0:/home/zyi# azcopy copy https://sastorageac0.blob.core.windows.net/sacontainer/1.zip https://eustorageac0.blob.core.windows.net/eucontainer/1.zip INFO: Scanning...INFO: Autologin not specified.INFO: Authenticating to destination using Azure ADINFO: Authenticating to source using Azure ADINFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder supportJob c4366598-5822-b946-45b9-6ee9d6f6df15 has startedLog file is located at: /root/.azcopy/c4366598-5822-b946-45b9-6ee9d6f6df15.logINFO: Transfers could fail because AzCopy could not verify if the destination supports tiers.100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total, Job c4366598-5822-b946-45b9-6ee9d6f6df15 summaryElapsed Time (Minutes): 4.6351Number of File Transfers: 1Number of Folder Property Transfers: 0Number of Symlink Transfers: 0Total Number of Transfers: 1Number of File Transfers Completed: 1Number of Folder Transfers Completed: 0Number of File Transfers Failed: 0Number of Folder Transfers Failed: 0Number of File Transfers Skipped: 0Number of Folder Transfers Skipped: 0TotalBytesTransferred: 10519930170Final Job Status: Completed#4root@ubuntu0:/home/zyi# azcopy copy https://sastorageac0.blob.core.windows.net/sacontainer/1.zip https://eustorageac0.blob.core.windows.net/eucontainer/1.zipINFO: Scanning...INFO: Autologin not specified.INFO: Authenticating to destination using Azure ADINFO: Authenticating to source using Azure ADINFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder supportJob 15fe8cca-78c3-3d40-4a55-e90d11495258 has startedLog file is located at: /root/.azcopy/15fe8cca-78c3-3d40-4a55-e90d11495258.logINFO: Transfers could fail because AzCopy could not verify if the destination supports tiers.100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total, Job 15fe8cca-78c3-3d40-4a55-e90d11495258 summaryElapsed Time (Minutes): 1.4338Number of File Transfers: 1Number of Folder Property Transfers: 0Number of Symlink Transfers: 0Total Number of Transfers: 1Number of File Transfers Completed: 1Number of Folder Transfers Completed: 0Number of File Transfers Failed: 0Number of Folder Transfers Failed: 0Number of File Transfers Skipped: 0Number of Folder Transfers Skipped: 0TotalBytesTransferred: 10519930170Final Job Status: Completed#5root@ubuntu0:/home/zyi# azcopy copy https://sastorageac0.blob.core.windows.net/sacontainer/1.zip https://eustorageac0.blob.core.windows.net/eucontainer/1.zipINFO: Scanning...INFO: Autologin not specified.INFO: Authenticating to destination using Azure ADINFO: Authenticating to source using Azure ADINFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder supportJob 2ac14c6f-f943-9c4b-7477-6947fd59ab31 has startedLog file is located at: /root/.azcopy/2ac14c6f-f943-9c4b-7477-6947fd59ab31.logINFO: Transfers could fail because AzCopy could not verify if the destination supports tiers.100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total, Job 2ac14c6f-f943-9c4b-7477-6947fd59ab31 summaryElapsed Time (Minutes): 1.8673Number of File Transfers: 1Number of Folder Property Transfers: 0Number of Symlink Transfers: 0Total Number of Transfers: 1Number of File Transfers Completed: 1Number of Folder Transfers Completed: 0Number of File Transfers Failed: 0Number of Folder Transfers Failed: 0Number of File Transfers Skipped: 0Number of Folder Transfers Skipped: 0TotalBytesTransferred: 10519930170Final Job Status: Completed```2. EU Contaner → SA Container```powershellroot@ubuntu0:/home/zyi# azcopy copy https://eustorageac0.blob.core.windows.net/eucontainer/10.zip https://sastorageac0.blob.core.windows.net/sacontainerINFO: Scanning...INFO: Autologin not specified.INFO: Authenticating to destination using Azure ADINFO: Authenticating to source using Azure ADINFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder supportJob 8b6f339f-e23b-2f4e-755c-8bd336942b40 has startedLog file is located at: /root/.azcopy/8b6f339f-e23b-2f4e-755c-8bd336942b40.logINFO: Transfers could fail because AzCopy could not verify if the destination supports tiers.100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total, Job 8b6f339f-e23b-2f4e-755c-8bd336942b40 summaryElapsed Time (Minutes): 1.0003Number of File Transfers: 1Number of Folder Property Transfers: 0Number of Symlink Transfers: 0Total Number of Transfers: 1Number of File Transfers Completed: 1Number of Folder Transfers Completed: 0Number of File Transfers Failed: 0Number of Folder Transfers Failed: 0Number of File Transfers Skipped: 0Number of Folder Transfers Skipped: 0TotalBytesTransferred: 10519930170Final Job Status: Completed#2root@ubuntu0:/home/zyi# azcopy copy https://eustorageac0.blob.core.windows.net/eucontainer/10.zip https://sastorageac0.blob.core.windows.net/sacontainer/1.zipINFO: Scanning...INFO: Autologin not specified.INFO: Authenticating to destination using Azure ADINFO: Authenticating to source using Azure ADINFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder supportJob 0c1039f0-7a9f-0e44-6a42-079c03708fb8 has startedLog file is located at: /root/.azcopy/0c1039f0-7a9f-0e44-6a42-079c03708fb8.logINFO: Transfers could fail because AzCopy could not verify if the destination supports tiers.100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total, Job 0c1039f0-7a9f-0e44-6a42-079c03708fb8 summaryElapsed Time (Minutes): 0.9336Number of File Transfers: 1Number of Folder Property Transfers: 0Number of Symlink Transfers: 0Total Number of Transfers: 1Number of File Transfers Completed: 1Number of Folder Transfers Completed: 0Number of File Transfers Failed: 0Number of Folder Transfers Failed: 0Number of File Transfers Skipped: 0Number of Folder Transfers Skipped: 0TotalBytesTransferred: 10519930170Final Job Status: Completed```1. EU client → EU Container```powershellroot@eu-ubuntu0:/home/zyi# azcopy copy 1.zip https://eustorageac0.blob.core.windows.net/eucontainer/1.zip INFO: Scanning...INFO: Autologin not specified.INFO: Authenticating to destination using Azure ADINFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder supportJob d3bdc10a-4142-8b41-7f9a-752c840c5f36 has startedLog file is located at: /root/.azcopy/d3bdc10a-4142-8b41-7f9a-752c840c5f36.log100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s): 623.3303Job d3bdc10a-4142-8b41-7f9a-752c840c5f36 summaryElapsed Time (Minutes): 0.767Number of File Transfers: 1Number of Folder Property Transfers: 0Number of Symlink Transfers: 0Total Number of Transfers: 1Number of File Transfers Completed: 1Number of Folder Transfers Completed: 0Number of File Transfers Failed: 0Number of Folder Transfers Failed: 0Number of File Transfers Skipped: 0Number of Folder Transfers Skipped: 0TotalBytesTransferred: 10519930170Final Job Status: Completed#2root@eu-ubuntu0:/home/zyi# azcopy copy 1.zip https://eustorageac0.blob.core.windows.net/eucontainer/1.zip INFO: Scanning...INFO: Autologin not specified.INFO: Authenticating to destination using Azure ADINFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder supportJob 3018151c-c777-ca42-4054-7f980de47650 has startedLog file is located at: /root/.azcopy/3018151c-c777-ca42-4054-7f980de47650.log100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s): 598.946 Job 3018151c-c777-ca42-4054-7f980de47650 summaryElapsed Time (Minutes): 0.7669Number of File Transfers: 1Number of Folder Property Transfers: 0Number of Symlink Transfers: 0Total Number of Transfers: 1Number of File Transfers Completed: 1Number of Folder Transfers Completed: 0Number of File Transfers Failed: 0Number of Folder Transfers Failed: 0Number of File Transfers Skipped: 0Number of Folder Transfers Skipped: 0TotalBytesTransferred: 10519930170Final Job Status: Completed#3root@eu-ubuntu0:/home/zyi# azcopy copy 1.zip https://eustorageac0.blob.core.windows.net/eucontainer/1.zip INFO: Scanning...INFO: Autologin not specified.INFO: Authenticating to destination using Azure ADINFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder supportJob 33cf320d-94a5-aa48-5a35-b94d34cf5cc1 has startedLog file is located at: /root/.azcopy/33cf320d-94a5-aa48-5a35-b94d34cf5cc1.log100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total, 2-sec Throughput (Mb/s): 579.6335Job 33cf320d-94a5-aa48-5a35-b94d34cf5cc1 summaryElapsed Time (Minutes): 0.767Number of File Transfers: 1Number of Folder Property Transfers: 0Number of Symlink Transfers: 0Total Number of Transfers: 1Number of File Transfers Completed: 1Number of Folder Transfers Completed: 0Number of File Transfers Failed: 0Number of Folder Transfers Failed: 0Number of File Transfers Skipped: 0Number of Folder Transfers Skipped: 0TotalBytesTransferred: 10519930170Final Job Status: Completed```2. EU Container → EU Client, VM disk iops limited```powershellroot@eu-ubuntu0:/home/zyi# azcopy copy https://eustorageac0.blob.core.windows.net/eucontainer/1.zip 10.zipINFO: Scanning...INFO: Autologin not specified.INFO: Authenticating to source using Azure ADINFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder supportJob 038b5c4a-7376-094d-4e7c-c958e15378f8 has startedLog file is located at: /root/.azcopy/038b5c4a-7376-094d-4e7c-c958e15378f8.log100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total, Job 038b5c4a-7376-094d-4e7c-c958e15378f8 summaryElapsed Time (Minutes): 0.8814Number of File Transfers: 1Number of Folder Property Transfers: 0Number of Symlink Transfers: 0Total Number of Transfers: 1Number of File Transfers Completed: 1Number of Folder Transfers Completed: 0Number of File Transfers Failed: 0Number of Folder Transfers Failed: 0Number of File Transfers Skipped: 0Number of Folder Transfers Skipped: 0TotalBytesTransferred: 10519930170Final Job Status: Completed```1. EA client →EU Container```powershellroot@ubuntu0:/home/zyi# azcopy copy 1.zip https://eustorageac0.blob.core.windows.net/eucontainerINFO: Scanning...INFO: Autologin not specified.INFO: Authenticating to destination using Azure ADINFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder supportJob e197a453-cd7d-9042-53bc-de6c0e26246c has startedLog file is located at: /root/.azcopy/e197a453-cd7d-9042-53bc-de6c0e26246c.log100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total, Job e197a453-cd7d-9042-53bc-de6c0e26246c summaryElapsed Time (Minutes): 0.5335Number of File Transfers: 1Number of Folder Property Transfers: 0Number of Symlink Transfers: 0Total Number of Transfers: 1Number of File Transfers Completed: 1Number of Folder Transfers Completed: 0Number of File Transfers Failed: 0Number of Folder Transfers Failed: 0Number of File Transfers Skipped: 0Number of Folder Transfers Skipped: 0TotalBytesTransferred: 10519930170Final Job Status: Completed#2root@ubuntu0:/home/zyi# azcopy copy 1.zip https://eustorageac0.blob.core.windows.net/eucontainer/10.zipINFO: Scanning...INFO: Autologin not specified.INFO: Authenticating to destination using Azure ADINFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder supportJob 6b1b50a2-3c6c-9c45-5165-085514255e76 has startedLog file is located at: /root/.azcopy/6b1b50a2-3c6c-9c45-5165-085514255e76.log100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total, Job 6b1b50a2-3c6c-9c45-5165-085514255e76 summaryElapsed Time (Minutes): 0.7002Number of File Transfers: 1Number of Folder Property Transfers: 0Number of Symlink Transfers: 0Total Number of Transfers: 1Number of File Transfers Completed: 1Number of Folder Transfers Completed: 0Number of File Transfers Failed: 0Number of Folder Transfers Failed: 0Number of File Transfers Skipped: 0Number of Folder Transfers Skipped: 0TotalBytesTransferred: 10519930170Final Job Status: Completed```2. EU container → EA container```powershellroot@ubuntu0:/home/zyi# azcopy copy https://eustorageac0.blob.core.windows.net/eucontainer/10.zip https://eastorageaccountpg.blob.core.windows.net/blog-pgINFO: Scanning...INFO: Autologin not specified.INFO: Authenticating to destination using Azure ADINFO: Authenticating to source using Azure ADINFO: Any empty folders will not be processed, because source and/or destination doesn't have full folder supportJob 04a69131-382f-3741-4263-c81b6e28c27e has startedLog file is located at: /root/.azcopy/04a69131-382f-3741-4263-c81b6e28c27e.logINFO: Transfers could fail because AzCopy could not verify if the destination supports tiers.100.0 %, 1 Done, 0 Failed, 0 Pending, 0 Skipped, 1 Total, Job 04a69131-382f-3741-4263-c81b6e28c27e summaryElapsed Time (Minutes): 0.4001Number of File Transfers: 1Number of Folder Property Transfers: 0Number of Symlink Transfers: 0Total Number of Transfers: 1Number of File Transfers Completed: 1Number of Folder Transfers Completed: 0Number of File Transfers Failed: 0Number of Folder Transfers Failed: 0Number of File Transfers Skipped: 0Number of Folder Transfers Skipped: 0TotalBytesTransferred: 10519930170Final Job Status: Completed```
相关文章：
Azure - AzCopy学习
使用 AzCopy 将本地数据迁移到云存储空间 azcopy login 创建存储账号 ./azcopy login --tenant-id 40242385-c249-4746-95dc-4a0b64d49dc5这里的—tenant-id 在下面的地方查看：目录 ID；需要拥有Storage Blob Data Owner 的权限账号下可能会有很多目录&am…...
解决无法远程连接MySQL服务的问题
① 设置MySQL中root用户的权限： [rootnginx-dev etc]# mysql -uroot -pRoot123 mysql> use mysql; mysql> GRANT ALL PRIVILEGES ON *.* TO root% IDENTIFIED BY Root123 WITH GRANT OPTION; mysql> select host,user,authentication_string from user; -…...
mybatiplus代码生成器
目录 1.pom文件引入 2.引入模板引擎 3.注意 新版本，老版本配置和用法都不太一样，此处暂不展示；另外也可以尝试一下MyBatis-Flex 总之mybatisplus有的或者收费的，它都有MyBatis-Flex 是什么 - MyBatis-Flex 官方网站 1.pom文件…...
leetcode分类刷题：哈希表（Hash Table）（三、循环存在问题）
1、当需要快速判断某元素是否出现在序列中时，就要用到哈希表了。 2、本文针对的总结题型为给定的序列或需要构造的序列中是否存在循环，与 160. 相交链表、 141. 环形链表、142. 环形链表 II的题型一样。 202. 快乐数 这道题还考察如何对正整数求解各个位…...
43、基于 springboot 自动配置的 spring mvc 错误处理，就是演示项目报错后，跳转到自定义的错误页面
Spring MVC 的错误处理：基于 SpringBoot 自动配置之后的 Spring MVC 错误处理。 就是访问方法时出错，然后弄个自定义的错误页面进行显示。 ★ 两种错误处理方式 方式一： 基于Spring Boot自动配置的错误处理方式，只要通过属性文件…...
干货分享，现代列式数据库系统如何设计与实现？ | StoneData 论文选读
作者：袁洋 | StoneData 技术架构师 审核：王博 论文链接：columnstoresfntdbs.pdf (harvard.edu) 列存四先驱和 MIT 知名教授 Samuel Madden 于 2013 年在某期刊上写的一篇当时列存相关技术的综述。文章还挺全面也很经典，通过剖析三…...
说说构建流批一体准实时数仓
分析&回答 基于 Hive 的离线数仓往往是企业大数据生产系统中不可缺少的一环。Hive 数仓有很高的成熟度和稳定性，但由于它是离线的，延时很大。在一些对延时要求比较高的场景，需要另外搭建基于 Flink 的实时数仓，将链路延时降低…...
北京筑龙受邀出席中物联“采购供应链中国行—走进雄安”活动
日前，“采购供应链中国行—走进雄安”活动在河北雄安新区成功举办，来自30家相关单位的50余名领导和代表参加了本次活动。活动由中国物流与采购联合会公共采购分会主办，中国物流与采购联合会采购委、中国雄安集团有限公司、河北雄安新区招标投…...
【Tkinter界面：练习-01】窗口-部件-布局
一、说明 python在用户界面开发中，其中有QT5，和Tkinter；对于实际项目，界面需要高大上，因此用QT5，对于开发人员的演示程序，或简单程序中，不建议QT5；用Tkinter已经足够。本…...
LeetCode每日一题：823. 带因子的二叉树（2023.8.29 C++）
目录 823. 带因子的二叉树 题目描述： 实现代码与解析： dp hash 原理思路： 823. 带因子的二叉树 题目描述： 给出一个含有不重复整数元素的数组 arr ，每个整数 arr[i] 均大于 1。 用这些整数来构建二叉树&#x…...
【教学类-35-01】学号+姓名+班级（描字帖）A4一页
背景说明： 本学期我带机动班，其中大4班去的频率比较高，与是我用大四班的名单做了一份 “描字帖”，在9月1日第一天见面时，孩子们用记号笔描字帖时，我也可以对这些孩子初步混个眼熟（聪明的&#x…...
UE5 里的一些常用的了解
# ACharacter、APawn的继承关系 ACharacter -继承自-> APawn -继承自-> AActor和 INavAgentInterface AActor -继承自-> UObject -继承自->UObjectBaseUtility -继承自-> UObjectBase（一个独立的类）INavAgentInterface是一个独立的类 #…...
【网络安全带你练爬虫-100练】第19练：使用python打开exe文件
目录 一、目标1：调用exe文件 二、目标2：调用exe打开文件 一、目标1：调用exe文件 1、subprocess 模块允许在 Python 中启动一个新的进程，并与其进行交互 2、subprocess.run() 函数来启动exe文件 3、subprocess.run(["文件路…...
【2D/3D RRT* 算法】使用快速探索随机树进行最佳路径规划（Matlab代码实现）
💥💥💞💞欢迎来到本博客❤️❤️💥💥 🏆博主优势：🌞🌞🌞博客内容尽量做到思维缜密，逻辑清晰，为了方便读者。 ⛳️座右铭&a…...
用反射实现自定义Java对象转化为json工具类
传入一个object类型的对象获取该对象的class类getFields方法获取该类的所有属性对属性进行遍历，并且拼接成Json格式的字符串，注意：通过属性名来推断方法名获取Method实例通过invoke方法调用 public static String objectToJsonUtil(Object o…...
rk3568 nvme硬盘分区，格式化，挂载测试
前言 环境介绍： 1.编译环境 Ubuntu 18.04.5 LTS 2.SDK rk356x_linux 3.单板 迅为itop-3568开发板 自制底板 一、查看硬盘 插上硬盘上电，进入系统后通过命令lspci查看nvme硬盘识别情况 [rootRK356X:/]# lspci -k 21:00.0 Class 0108: 1e4b:1202…...
Failed to load ApplicationContext解决办法，spring版本问题
有如下报错： "D:\Program Files\Java\jdk-13.0.1\bin\java.exe" -agentlib:jdwptransportdt_socket,address127.0.0.1:7325,suspendy,servern -ea -Didea.test.cyclic.buffer.size1048576 -Dfile.encodingUTF-8 -classpath "D:\Program Files\JetBr…...
Is f(z)=1/z truly an analytic function
https://math.stackexchange.com/questions/755566/is-fz-1-z-truly-an-analytic-function...
代理模式 静态代理和动态代理（jdk、cglib）——Java入职第十一天
一、代理模式 一个类代表另一个类去完成扩展功能，在主体类的基础上，新增一个代理类，扩展主体类功能，不影响主体，完成额外功能。比如买车票，可以去代理点买，不用去火车站，主要包括静态代理和动态代理两种模式。 代理类中包含了主体类 二、静态代理 无法根据业务扩展，…...
Remmina在ubuntu22.04中无法连接Windows
Remmina在ubuntu22.04中无法连接Windows 问题 提示为： 无法通过TLS到RDP服务器… 分析 原因是Remmina需要使用openssl通过RDP加密与Windows计算机连接，而ubuntu22.04系统中OpenSSL版本为3.0，Openssl3 将 tls<1.2 和 sha1 的默认安全级别…...
【大模型RAG】拍照搜题技术架构速览：三层管道、两级检索、兜底大模型
摘要 拍照搜题系统采用“三层管道（多模态 OCR → 语义检索 → 答案渲染）、两级检索（倒排 BM25 向量 HNSW）并以大语言模型兜底”的整体框架： 多模态 OCR 层 将题目图片经过超分、去噪、倾斜校正后，分别用…...
k8s从入门到放弃之Ingress七层负载
k8s从入门到放弃之Ingress七层负载 在Kubernetes（简称K8s）中，Ingress是一个API对象，它允许你定义如何从集群外部访问集群内部的服务。Ingress可以提供负载均衡、SSL终结和基于名称的虚拟主机等功能。通过Ingress，你可…...
关于nvm与node.js
1 安装nvm 安装过程中手动修改 nvm的安装路径， 以及修改 通过nvm安装node后正在使用的node的存放目录【这句话可能难以理解，但接着往下看你就了然了】 2 修改nvm中settings.txt文件配置 nvm安装成功后，通常在该文件中会出现以下配置&…...
Linux相关概念和易错知识点（42）（TCP的连接管理、可靠性、面临复杂网络的处理）
目录 1.TCP的连接管理机制（1）三次握手①握手过程②对握手过程的理解 （2）四次挥手（3）握手和挥手的触发（4）状态切换①挥手过程中状态的切换②握手过程中状态的切换 2.TCP的可靠性&…...
CMake 从 GitHub 下载第三方库并使用
有时我们希望直接使用 GitHub 上的开源库，而不想手动下载、编译和安装。 可以利用 CMake 提供的 FetchContent 模块来实现自动下载、构建和链接第三方库。 FetchContent 命令官方文档✅ 示例代码 我们将以 fmt 这个流行的格式化库为例，演示如何： 使用 FetchContent 从 GitH…...
数据库分批入库
今天在工作中，遇到一个问题，就是分批查询的时候，由于批次过大导致出现了一些问题，一下是问题描述和解决方案： 示例： // 假设已有数据列表 dataList 和 PreparedStatement pstmt int batchSize 1000; // …...
鸿蒙DevEco Studio HarmonyOS 5跑酷小游戏实现指南
1. 项目概述 本跑酷小游戏基于鸿蒙HarmonyOS 5开发，使用DevEco Studio作为开发工具，采用Java语言实现，包含角色控制、障碍物生成和分数计算系统。 2. 项目结构 /src/main/java/com/example/runner/├── MainAbilitySlice.java // 主界…...
代理篇12|深入理解 Vite中的Proxy接口代理配置
在前端开发中，常常会遇到 跨域请求接口 的情况。为了解决这个问题，Vite 和 Webpack 都提供了 proxy 代理功能，用于将本地开发请求转发到后端服务器。 什么是代理（proxy）？ 代理是在开发过程中，前端项目通过开发服务器，将指定的请求“转发”到真实的后端服务器，从而绕…...
QT3D学习笔记——圆台、圆锥
类名作用Qt3DWindow3D渲染窗口容器QEntity场景中的实体（对象或容器）QCamera控制观察视角QPointLight点光源QConeMesh圆锥几何网格QTransform控制实体的位置/旋转/缩放QPhongMaterialPhong光照材质（定义颜色、反光等）QFirstPersonC…...
GruntJS-前端自动化任务运行器从入门到实战
Grunt 完全指南：从入门到实战 一、Grunt 是什么？ Grunt是一个基于 Node.js 的前端自动化任务运行器，主要用于自动化执行项目开发中重复性高的任务，例如文件压缩、代码编译、语法检查、单元测试、文件合并等。通过配置简洁的任务…...