Started by timer
Running as SYSTEM
Building in workspace /var/lib/jenkins/jobs/tensorflow/workspace
[SSH] script:
module load anaconda3_gpu
module load cudnn
cd python
rm -f tensorflow_test.txt
srun --nodes=1 --ntasks-per-node=1 --mem=16g \
--time=00:05:00 \
--partition=a100rome \
--account=bbmb-hydro \
--gpus-per-node=1 \
--gpu-bind=closest \
python3 cifar10gpu.py | tee tensorflow_test.txt
YVALUE=`grep "wall time seconds" tensorflow_test.txt | cut -d'=' -f2`
echo YVALUE=$YVALUE > tensorflow_time.txt
[SSH] executing...
srun: job 96143 queued and waiting for resources
srun: job 96143 has been allocated resources
2024-10-31 07:43:07.658130: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-10-31 07:43:32.884883: E tensorflow/stream_executor/cuda/cuda_blas.cc:2981] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-10-31 07:44:06.006954: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /sw/spack/hydrogpu-2022-06/apps/cudnn/8.4.0.27-11.6-gcc-11.3.0-qbhsfav/lib:/sw/spack/hydrogpu-2022-06/apps/cuda/11.7.0-gcc-11.3.0-3ysno6b/lib64:/sw/spack/hydrogpu-2022-06/apps/openmpi/4.1.4-gcc-11.3.0-i6koipj/lib:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib64:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib
2024-10-31 07:44:06.007582: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /sw/spack/hydrogpu-2022-06/apps/cudnn/8.4.0.27-11.6-gcc-11.3.0-qbhsfav/lib:/sw/spack/hydrogpu-2022-06/apps/cuda/11.7.0-gcc-11.3.0-3ysno6b/lib64:/sw/spack/hydrogpu-2022-06/apps/openmpi/4.1.4-gcc-11.3.0-i6koipj/lib:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib64:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib
2024-10-31 07:44:06.007590: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
2024-10-31 07:44:58.224922: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-10-31 07:45:01.244938: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1616] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 79148 MB memory: -> device: 0, name: NVIDIA A100 80GB PCIe, pci bus id: 0000:21:00.0, compute capability: 8.0
(50000, 32, 32, 3) (10000, 32, 32, 3)
Epoch 1/10
2024-10-31 07:45:08.210798: I tensorflow/stream_executor/cuda/cuda_blas.cc:1614] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
1/1563 [..............................] - ETA: 1:40:19 - loss: 2.4201 - accuracy: 0.1250
37/1563 [..............................] - ETA: 2s - loss: 2.2345 - accuracy: 0.1689
74/1563 [>.............................] - ETA: 2s - loss: 2.1828 - accuracy: 0.1964
112/1563 [=>............................] - ETA: 1s - loss: 2.1350 - accuracy: 0.2201
150/1563 [=>............................] - ETA: 1s - loss: 2.0990 - accuracy: 0.2352
188/1563 [==>...........................] - ETA: 1s - loss: 2.0762 - accuracy: 0.2497
226/1563 [===>..........................] - ETA: 1s - loss: 2.0525 - accuracy: 0.2618
264/1563 [====>.........................] - ETA: 1s - loss: 2.0328 - accuracy: 0.2685
302/1563 [====>.........................] - ETA: 1s - loss: 2.0084 - accuracy: 0.2791
340/1563 [=====>........................] - ETA: 1s - loss: 1.9932 - accuracy: 0.2837
378/1563 [======>.......................] - ETA: 1s - loss: 1.9833 - accuracy: 0.2884
416/1563 [======>.......................] - ETA: 1s - loss: 1.9720 - accuracy: 0.2927
453/1563 [=======>......................] - ETA: 1s - loss: 1.9632 - accuracy: 0.2961
490/1563 [========>.....................] - ETA: 1s - loss: 1.9522 - accuracy: 0.3013
527/1563 [=========>....................] - ETA: 1s - loss: 1.9455 - accuracy: 0.3029
565/1563 [=========>....................] - ETA: 1s - loss: 1.9375 - accuracy: 0.3055
603/1563 [==========>...................] - ETA: 1s - loss: 1.9328 - accuracy: 0.3080
641/1563 [===========>..................] - ETA: 1s - loss: 1.9251 - accuracy: 0.3113
678/1563 [============>.................] - ETA: 1s - loss: 1.9149 - accuracy: 0.3157
716/1563 [============>.................] - ETA: 1s - loss: 1.9063 - accuracy: 0.3188
754/1563 [=============>................] - ETA: 1s - loss: 1.9004 - accuracy: 0.3207
792/1563 [==============>...............] - ETA: 1s - loss: 1.8957 - accuracy: 0.3227
829/1563 [==============>...............] - ETA: 0s - loss: 1.8921 - accuracy: 0.3245
866/1563 [===============>..............] - ETA: 0s - loss: 1.8860 - accuracy: 0.3270
903/1563 [================>.............] - ETA: 0s - loss: 1.8797 - accuracy: 0.3292
940/1563 [=================>............] - ETA: 0s - loss: 1.8751 - accuracy: 0.3315
978/1563 [=================>............] - ETA: 0s - loss: 1.8696 - accuracy: 0.3333
1016/1563 [==================>...........] - ETA: 0s - loss: 1.8639 - accuracy: 0.3356
1054/1563 [===================>..........] - ETA: 0s - loss: 1.8600 - accuracy: 0.3371
1092/1563 [===================>..........] - ETA: 0s - loss: 1.8539 - accuracy: 0.3401
1130/1563 [====================>.........] - ETA: 0s - loss: 1.8513 - accuracy: 0.3410
1168/1563 [=====================>........] - ETA: 0s - loss: 1.8452 - accuracy: 0.3429
1206/1563 [======================>.......] - ETA: 0s - loss: 1.8411 - accuracy: 0.3447
1244/1563 [======================>.......] - ETA: 0s - loss: 1.8371 - accuracy: 0.3462
1282/1563 [=======================>......] - ETA: 0s - loss: 1.8357 - accuracy: 0.3467
1320/1563 [========================>.....] - ETA: 0s - loss: 1.8321 - accuracy: 0.3480
1358/1563 [=========================>....] - ETA: 0s - loss: 1.8301 - accuracy: 0.3484
1396/1563 [=========================>....] - ETA: 0s - loss: 1.8258 - accuracy: 0.3502
1434/1563 [==========================>...] - ETA: 0s - loss: 1.8220 - accuracy: 0.3513
1472/1563 [===========================>..] - ETA: 0s - loss: 1.8174 - accuracy: 0.3533
1510/1563 [===========================>..] - ETA: 0s - loss: 1.8140 - accuracy: 0.3542
1548/1563 [============================>.] - ETA: 0s - loss: 1.8114 - accuracy: 0.3552
1563/1563 [==============================] - 6s 1ms/step - loss: 1.8107 - accuracy: 0.3555
Epoch 2/10
1/1563 [..............................] - ETA: 2s - loss: 1.5013 - accuracy: 0.4062
38/1563 [..............................] - ETA: 2s - loss: 1.6619 - accuracy: 0.4186
76/1563 [>.............................] - ETA: 2s - loss: 1.6602 - accuracy: 0.4178
114/1563 [=>............................] - ETA: 1s - loss: 1.6563 - accuracy: 0.4208
152/1563 [=>............................] - ETA: 1s - loss: 1.6625 - accuracy: 0.4188
190/1563 [==>...........................] - ETA: 1s - loss: 1.6642 - accuracy: 0.4163
228/1563 [===>..........................] - ETA: 1s - loss: 1.6577 - accuracy: 0.4171
266/1563 [====>.........................] - ETA: 1s - loss: 1.6546 - accuracy: 0.4194
304/1563 [====>.........................] - ETA: 1s - loss: 1.6523 - accuracy: 0.4184
342/1563 [=====>........................] - ETA: 1s - loss: 1.6520 - accuracy: 0.4174
380/1563 [======>.......................] - ETA: 1s - loss: 1.6535 - accuracy: 0.4174
418/1563 [=======>......................] - ETA: 1s - loss: 1.6541 - accuracy: 0.4172
456/1563 [=======>......................] - ETA: 1s - loss: 1.6570 - accuracy: 0.4161
494/1563 [========>.....................] - ETA: 1s - loss: 1.6565 - accuracy: 0.4161
532/1563 [=========>....................] - ETA: 1s - loss: 1.6545 - accuracy: 0.4175
570/1563 [=========>....................] - ETA: 1s - loss: 1.6499 - accuracy: 0.4189
607/1563 [==========>...................] - ETA: 1s - loss: 1.6481 - accuracy: 0.4195
645/1563 [===========>..................] - ETA: 1s - loss: 1.6462 - accuracy: 0.4195
683/1563 [============>.................] - ETA: 1s - loss: 1.6453 - accuracy: 0.4194
721/1563 [============>.................] - ETA: 1s - loss: 1.6441 - accuracy: 0.4196
759/1563 [=============>................] - ETA: 1s - loss: 1.6415 - accuracy: 0.4217
797/1563 [==============>...............] - ETA: 1s - loss: 1.6415 - accuracy: 0.4217
835/1563 [===============>..............] - ETA: 0s - loss: 1.6414 - accuracy: 0.4222
873/1563 [===============>..............] - ETA: 0s - loss: 1.6413 - accuracy: 0.4217
911/1563 [================>.............] - ETA: 0s - loss: 1.6418 - accuracy: 0.4215
949/1563 [=================>............] - ETA: 0s - loss: 1.6399 - accuracy: 0.4222
986/1563 [=================>............] - ETA: 0s - loss: 1.6375 - accuracy: 0.4228
1024/1563 [==================>...........] - ETA: 0s - loss: 1.6369 - accuracy: 0.4229
1061/1563 [===================>..........] - ETA: 0s - loss: 1.6343 - accuracy: 0.4234
1099/1563 [====================>.........] - ETA: 0s - loss: 1.6344 - accuracy: 0.4230
1137/1563 [====================>.........] - ETA: 0s - loss: 1.6315 - accuracy: 0.4243
1175/1563 [=====================>........] - ETA: 0s - loss: 1.6304 - accuracy: 0.4247
1213/1563 [======================>.......] - ETA: 0s - loss: 1.6308 - accuracy: 0.4249
1251/1563 [=======================>......] - ETA: 0s - loss: 1.6300 - accuracy: 0.4253
1289/1563 [=======================>......] - ETA: 0s - loss: 1.6290 - accuracy: 0.4254
1327/1563 [========================>.....] - ETA: 0s - loss: 1.6274 - accuracy: 0.4257
1364/1563 [=========================>....] - ETA: 0s - loss: 1.6273 - accuracy: 0.4254
1401/1563 [=========================>....] - ETA: 0s - loss: 1.6262 - accuracy: 0.4258
1438/1563 [==========================>...] - ETA: 0s - loss: 1.6243 - accuracy: 0.4264
1476/1563 [===========================>..] - ETA: 0s - loss: 1.6236 - accuracy: 0.4267
1514/1563 [============================>.] - ETA: 0s - loss: 1.6230 - accuracy: 0.4271
1552/1563 [============================>.] - ETA: 0s - loss: 1.6214 - accuracy: 0.4275
1563/1563 [==============================] - 2s 1ms/step - loss: 1.6210 - accuracy: 0.4278
Epoch 3/10
1/1563 [..............................] - ETA: 2s - loss: 1.8090 - accuracy: 0.2812
39/1563 [..............................] - ETA: 2s - loss: 1.5751 - accuracy: 0.4383
77/1563 [>.............................] - ETA: 1s - loss: 1.5768 - accuracy: 0.4501
115/1563 [=>............................] - ETA: 1s - loss: 1.5629 - accuracy: 0.4527
153/1563 [=>............................] - ETA: 1s - loss: 1.5600 - accuracy: 0.4471
191/1563 [==>...........................] - ETA: 1s - loss: 1.5536 - accuracy: 0.4504
228/1563 [===>..........................] - ETA: 1s - loss: 1.5508 - accuracy: 0.4527
266/1563 [====>.........................] - ETA: 1s - loss: 1.5553 - accuracy: 0.4474
303/1563 [====>.........................] - ETA: 1s - loss: 1.5545 - accuracy: 0.4483
340/1563 [=====>........................] - ETA: 1s - loss: 1.5579 - accuracy: 0.4476
378/1563 [======>.......................] - ETA: 1s - loss: 1.5541 - accuracy: 0.4489
416/1563 [======>.......................] - ETA: 1s - loss: 1.5559 - accuracy: 0.4512
454/1563 [=======>......................] - ETA: 1s - loss: 1.5576 - accuracy: 0.4496
491/1563 [========>.....................] - ETA: 1s - loss: 1.5601 - accuracy: 0.4515
527/1563 [=========>....................] - ETA: 1s - loss: 1.5594 - accuracy: 0.4529
565/1563 [=========>....................] - ETA: 1s - loss: 1.5587 - accuracy: 0.4539
603/1563 [==========>...................] - ETA: 1s - loss: 1.5594 - accuracy: 0.4533
641/1563 [===========>..................] - ETA: 1s - loss: 1.5605 - accuracy: 0.4512
679/1563 [============>.................] - ETA: 1s - loss: 1.5607 - accuracy: 0.4514
717/1563 [============>.................] - ETA: 1s - loss: 1.5561 - accuracy: 0.4526
755/1563 [=============>................] - ETA: 1s - loss: 1.5568 - accuracy: 0.4524
793/1563 [==============>...............] - ETA: 1s - loss: 1.5552 - accuracy: 0.4532
831/1563 [==============>...............] - ETA: 0s - loss: 1.5539 - accuracy: 0.4538
869/1563 [===============>..............] - ETA: 0s - loss: 1.5528 - accuracy: 0.4544
907/1563 [================>.............] - ETA: 0s - loss: 1.5517 - accuracy: 0.4552
945/1563 [=================>............] - ETA: 0s - loss: 1.5483 - accuracy: 0.4561
983/1563 [=================>............] - ETA: 0s - loss: 1.5462 - accuracy: 0.4565
1020/1563 [==================>...........] - ETA: 0s - loss: 1.5455 - accuracy: 0.4559
1058/1563 [===================>..........] - ETA: 0s - loss: 1.5450 - accuracy: 0.4557
1096/1563 [====================>.........] - ETA: 0s - loss: 1.5448 - accuracy: 0.4561
1134/1563 [====================>.........] - ETA: 0s - loss: 1.5462 - accuracy: 0.4554
1172/1563 [=====================>........] - ETA: 0s - loss: 1.5474 - accuracy: 0.4555
1210/1563 [======================>.......] - ETA: 0s - loss: 1.5463 - accuracy: 0.4556
1248/1563 [======================>.......] - ETA: 0s - loss: 1.5453 - accuracy: 0.4561
1286/1563 [=======================>......] - ETA: 0s - loss: 1.5460 - accuracy: 0.4556
1324/1563 [========================>.....] - ETA: 0s - loss: 1.5451 - accuracy: 0.4556
1361/1563 [=========================>....] - ETA: 0s - loss: 1.5456 - accuracy: 0.4556
1399/1563 [=========================>....] - ETA: 0s - loss: 1.5462 - accuracy: 0.4554
1437/1563 [==========================>...] - ETA: 0s - loss: 1.5452 - accuracy: 0.4561
1475/1563 [===========================>..] - ETA: 0s - loss: 1.5443 - accuracy: 0.4571
1513/1563 [============================>.] - ETA: 0s - loss: 1.5426 - accuracy: 0.4579
1551/1563 [============================>.] - ETA: 0s - loss: 1.5410 - accuracy: 0.4583
1563/1563 [==============================] - 2s 1ms/step - loss: 1.5409 - accuracy: 0.4584
Epoch 4/10
1/1563 [..............................] - ETA: 2s - loss: 1.4209 - accuracy: 0.5625
38/1563 [..............................] - ETA: 2s - loss: 1.4527 - accuracy: 0.4819
76/1563 [>.............................] - ETA: 2s - loss: 1.4919 - accuracy: 0.4807
114/1563 [=>............................] - ETA: 1s - loss: 1.5002 - accuracy: 0.4734
152/1563 [=>............................] - ETA: 1s - loss: 1.4916 - accuracy: 0.4786
190/1563 [==>...........................] - ETA: 1s - loss: 1.4984 - accuracy: 0.4752
228/1563 [===>..........................] - ETA: 1s - loss: 1.5009 - accuracy: 0.4749
266/1563 [====>.........................] - ETA: 1s - loss: 1.5057 - accuracy: 0.4726
304/1563 [====>.........................] - ETA: 1s - loss: 1.5025 - accuracy: 0.4746
342/1563 [=====>........................] - ETA: 1s - loss: 1.4987 - accuracy: 0.4768
380/1563 [======>.......................] - ETA: 1s - loss: 1.4986 - accuracy: 0.4764
418/1563 [=======>......................] - ETA: 1s - loss: 1.4976 - accuracy: 0.4757
456/1563 [=======>......................] - ETA: 1s - loss: 1.4973 - accuracy: 0.4743
494/1563 [========>.....................] - ETA: 1s - loss: 1.4951 - accuracy: 0.4746
532/1563 [=========>....................] - ETA: 1s - loss: 1.4958 - accuracy: 0.4740
570/1563 [=========>....................] - ETA: 1s - loss: 1.4934 - accuracy: 0.4738
608/1563 [==========>...................] - ETA: 1s - loss: 1.4916 - accuracy: 0.4737
645/1563 [===========>..................] - ETA: 1s - loss: 1.4911 - accuracy: 0.4734
683/1563 [============>.................] - ETA: 1s - loss: 1.4922 - accuracy: 0.4731
721/1563 [============>.................] - ETA: 1s - loss: 1.4907 - accuracy: 0.4732
759/1563 [=============>................] - ETA: 1s - loss: 1.4900 - accuracy: 0.4733
797/1563 [==============>...............] - ETA: 1s - loss: 1.4892 - accuracy: 0.4747
835/1563 [===============>..............] - ETA: 0s - loss: 1.4872 - accuracy: 0.4756
873/1563 [===============>..............] - ETA: 0s - loss: 1.4878 - accuracy: 0.4759
911/1563 [================>.............] - ETA: 0s - loss: 1.4881 - accuracy: 0.4763
949/1563 [=================>............] - ETA: 0s - loss: 1.4866 - accuracy: 0.4767
987/1563 [=================>............] - ETA: 0s - loss: 1.4860 - accuracy: 0.4772
1025/1563 [==================>...........] - ETA: 0s - loss: 1.4857 - accuracy: 0.4772
1063/1563 [===================>..........] - ETA: 0s - loss: 1.4842 - accuracy: 0.4773
1101/1563 [====================>.........] - ETA: 0s - loss: 1.4833 - accuracy: 0.4774
1139/1563 [====================>.........] - ETA: 0s - loss: 1.4836 - accuracy: 0.4776
1177/1563 [=====================>........] - ETA: 0s - loss: 1.4838 - accuracy: 0.4780
1214/1563 [======================>.......] - ETA: 0s - loss: 1.4835 - accuracy: 0.4782
1251/1563 [=======================>......] - ETA: 0s - loss: 1.4854 - accuracy: 0.4776
1289/1563 [=======================>......] - ETA: 0s - loss: 1.4831 - accuracy: 0.4781
1326/1563 [========================>.....] - ETA: 0s - loss: 1.4822 - accuracy: 0.4786
1364/1563 [=========================>....] - ETA: 0s - loss: 1.4816 - accuracy: 0.4784
1402/1563 [=========================>....] - ETA: 0s - loss: 1.4802 - accuracy: 0.4788
1440/1563 [==========================>...] - ETA: 0s - loss: 1.4792 - accuracy: 0.4793
1477/1563 [===========================>..] - ETA: 0s - loss: 1.4797 - accuracy: 0.4791
1514/1563 [============================>.] - ETA: 0s - loss: 1.4789 - accuracy: 0.4794
1552/1563 [============================>.] - ETA: 0s - loss: 1.4790 - accuracy: 0.4795
1563/1563 [==============================] - 2s 1ms/step - loss: 1.4792 - accuracy: 0.4794
Epoch 5/10
1/1563 [..............................] - ETA: 2s - loss: 1.5872 - accuracy: 0.4062
39/1563 [..............................] - ETA: 2s - loss: 1.4042 - accuracy: 0.4928
77/1563 [>.............................] - ETA: 2s - loss: 1.4165 - accuracy: 0.5024
115/1563 [=>............................] - ETA: 1s - loss: 1.4103 - accuracy: 0.5035
153/1563 [=>............................] - ETA: 1s - loss: 1.4198 - accuracy: 0.5039
191/1563 [==>...........................] - ETA: 1s - loss: 1.4250 - accuracy: 0.5016
228/1563 [===>..........................] - ETA: 1s - loss: 1.4237 - accuracy: 0.5014
265/1563 [====>.........................] - ETA: 1s - loss: 1.4284 - accuracy: 0.4998
303/1563 [====>.........................] - ETA: 1s - loss: 1.4342 - accuracy: 0.4974
340/1563 [=====>........................] - ETA: 1s - loss: 1.4365 - accuracy: 0.4974
378/1563 [======>.......................] - ETA: 1s - loss: 1.4349 - accuracy: 0.4983
416/1563 [======>.......................] - ETA: 1s - loss: 1.4331 - accuracy: 0.4969
454/1563 [=======>......................] - ETA: 1s - loss: 1.4337 - accuracy: 0.4974
492/1563 [========>.....................] - ETA: 1s - loss: 1.4348 - accuracy: 0.4952
530/1563 [=========>....................] - ETA: 1s - loss: 1.4348 - accuracy: 0.4962
568/1563 [=========>....................] - ETA: 1s - loss: 1.4328 - accuracy: 0.4966
606/1563 [==========>...................] - ETA: 1s - loss: 1.4325 - accuracy: 0.4967
644/1563 [===========>..................] - ETA: 1s - loss: 1.4319 - accuracy: 0.4968
682/1563 [============>.................] - ETA: 1s - loss: 1.4300 - accuracy: 0.4974
720/1563 [============>.................] - ETA: 1s - loss: 1.4300 - accuracy: 0.4968
758/1563 [=============>................] - ETA: 1s - loss: 1.4309 - accuracy: 0.4962
796/1563 [==============>...............] - ETA: 1s - loss: 1.4312 - accuracy: 0.4963
834/1563 [===============>..............] - ETA: 0s - loss: 1.4322 - accuracy: 0.4957
872/1563 [===============>..............] - ETA: 0s - loss: 1.4320 - accuracy: 0.4953
910/1563 [================>.............] - ETA: 0s - loss: 1.4321 - accuracy: 0.4949
947/1563 [=================>............] - ETA: 0s - loss: 1.4304 - accuracy: 0.4964
984/1563 [=================>............] - ETA: 0s - loss: 1.4302 - accuracy: 0.4961
1021/1563 [==================>...........] - ETA: 0s - loss: 1.4313 - accuracy: 0.4954
1058/1563 [===================>..........] - ETA: 0s - loss: 1.4322 - accuracy: 0.4953
1096/1563 [====================>.........] - ETA: 0s - loss: 1.4324 - accuracy: 0.4945
1134/1563 [====================>.........] - ETA: 0s - loss: 1.4311 - accuracy: 0.4949
1172/1563 [=====================>........] - ETA: 0s - loss: 1.4327 - accuracy: 0.4939
1210/1563 [======================>.......] - ETA: 0s - loss: 1.4341 - accuracy: 0.4934
1247/1563 [======================>.......] - ETA: 0s - loss: 1.4348 - accuracy: 0.4929
1285/1563 [=======================>......] - ETA: 0s - loss: 1.4349 - accuracy: 0.4928
1323/1563 [========================>.....] - ETA: 0s - loss: 1.4355 - accuracy: 0.4930
1360/1563 [=========================>....] - ETA: 0s - loss: 1.4347 - accuracy: 0.4931
1398/1563 [=========================>....] - ETA: 0s - loss: 1.4337 - accuracy: 0.4938
1436/1563 [==========================>...] - ETA: 0s - loss: 1.4337 - accuracy: 0.4936
1474/1563 [===========================>..] - ETA: 0s - loss: 1.4325 - accuracy: 0.4944
1512/1563 [============================>.] - ETA: 0s - loss: 1.4330 - accuracy: 0.4942
1550/1563 [============================>.] - ETA: 0s - loss: 1.4320 - accuracy: 0.4947
1563/1563 [==============================] - 2s 1ms/step - loss: 1.4315 - accuracy: 0.4949
Epoch 6/10
1/1563 [..............................] - ETA: 2s - loss: 1.3437 - accuracy: 0.5000
38/1563 [..............................] - ETA: 2s - loss: 1.3705 - accuracy: 0.5181
75/1563 [>.............................] - ETA: 2s - loss: 1.3786 - accuracy: 0.5146
113/1563 [=>............................] - ETA: 1s - loss: 1.3808 - accuracy: 0.5122
151/1563 [=>............................] - ETA: 1s - loss: 1.3895 - accuracy: 0.5072
189/1563 [==>...........................] - ETA: 1s - loss: 1.3919 - accuracy: 0.5081
227/1563 [===>..........................] - ETA: 1s - loss: 1.3957 - accuracy: 0.5105
265/1563 [====>.........................] - ETA: 1s - loss: 1.3975 - accuracy: 0.5098
303/1563 [====>.........................] - ETA: 1s - loss: 1.3980 - accuracy: 0.5102
341/1563 [=====>........................] - ETA: 1s - loss: 1.3941 - accuracy: 0.5126
379/1563 [======>.......................] - ETA: 1s - loss: 1.3937 - accuracy: 0.5123
417/1563 [=======>......................] - ETA: 1s - loss: 1.3943 - accuracy: 0.5105
455/1563 [=======>......................] - ETA: 1s - loss: 1.3955 - accuracy: 0.5104
493/1563 [========>.....................] - ETA: 1s - loss: 1.3926 - accuracy: 0.5115
531/1563 [=========>....................] - ETA: 1s - loss: 1.3932 - accuracy: 0.5104
569/1563 [=========>....................] - ETA: 1s - loss: 1.3929 - accuracy: 0.5103
607/1563 [==========>...................] - ETA: 1s - loss: 1.3914 - accuracy: 0.5099
645/1563 [===========>..................] - ETA: 1s - loss: 1.3892 - accuracy: 0.5113
683/1563 [============>.................] - ETA: 1s - loss: 1.3923 - accuracy: 0.5092
721/1563 [============>.................] - ETA: 1s - loss: 1.3906 - accuracy: 0.5097
759/1563 [=============>................] - ETA: 1s - loss: 1.3893 - accuracy: 0.5105
797/1563 [==============>...............] - ETA: 1s - loss: 1.3902 - accuracy: 0.5109
835/1563 [===============>..............] - ETA: 0s - loss: 1.3877 - accuracy: 0.5112
873/1563 [===============>..............] - ETA: 0s - loss: 1.3873 - accuracy: 0.5118
910/1563 [================>.............] - ETA: 0s - loss: 1.3851 - accuracy: 0.5123
948/1563 [=================>............] - ETA: 0s - loss: 1.3875 - accuracy: 0.5116
985/1563 [=================>............] - ETA: 0s - loss: 1.3874 - accuracy: 0.5115
1023/1563 [==================>...........] - ETA: 0s - loss: 1.3879 - accuracy: 0.5113
1061/1563 [===================>..........] - ETA: 0s - loss: 1.3881 - accuracy: 0.5115
1099/1563 [====================>.........] - ETA: 0s - loss: 1.3886 - accuracy: 0.5116
1137/1563 [====================>.........] - ETA: 0s - loss: 1.3902 - accuracy: 0.5115
1175/1563 [=====================>........] - ETA: 0s - loss: 1.3888 - accuracy: 0.5120
1213/1563 [======================>.......] - ETA: 0s - loss: 1.3896 - accuracy: 0.5112
1251/1563 [=======================>......] - ETA: 0s - loss: 1.3890 - accuracy: 0.5115
1289/1563 [=======================>......] - ETA: 0s - loss: 1.3913 - accuracy: 0.5109
1327/1563 [========================>.....] - ETA: 0s - loss: 1.3927 - accuracy: 0.5107
1365/1563 [=========================>....] - ETA: 0s - loss: 1.3907 - accuracy: 0.5113
1403/1563 [=========================>....] - ETA: 0s - loss: 1.3910 - accuracy: 0.5111
1441/1563 [==========================>...] - ETA: 0s - loss: 1.3908 - accuracy: 0.5116
1479/1563 [===========================>..] - ETA: 0s - loss: 1.3900 - accuracy: 0.5122
1516/1563 [============================>.] - ETA: 0s - loss: 1.3893 - accuracy: 0.5128
1553/1563 [============================>.] - ETA: 0s - loss: 1.3897 - accuracy: 0.5131
1563/1563 [==============================] - 2s 1ms/step - loss: 1.3895 - accuracy: 0.5132
Epoch 7/10
1/1563 [..............................] - ETA: 2s - loss: 1.1149 - accuracy: 0.6250
39/1563 [..............................] - ETA: 2s - loss: 1.3790 - accuracy: 0.4968
76/1563 [>.............................] - ETA: 2s - loss: 1.3333 - accuracy: 0.5189
114/1563 [=>............................] - ETA: 1s - loss: 1.3295 - accuracy: 0.5269
152/1563 [=>............................] - ETA: 1s - loss: 1.3448 - accuracy: 0.5226
190/1563 [==>...........................] - ETA: 1s - loss: 1.3459 - accuracy: 0.5245
228/1563 [===>..........................] - ETA: 1s - loss: 1.3473 - accuracy: 0.5260
265/1563 [====>.........................] - ETA: 1s - loss: 1.3436 - accuracy: 0.5289
302/1563 [====>.........................] - ETA: 1s - loss: 1.3406 - accuracy: 0.5293
339/1563 [=====>........................] - ETA: 1s - loss: 1.3441 - accuracy: 0.5286
376/1563 [======>.......................] - ETA: 1s - loss: 1.3482 - accuracy: 0.5277
414/1563 [======>.......................] - ETA: 1s - loss: 1.3536 - accuracy: 0.5265
452/1563 [=======>......................] - ETA: 1s - loss: 1.3554 - accuracy: 0.5245
490/1563 [========>.....................] - ETA: 1s - loss: 1.3560 - accuracy: 0.5246
528/1563 [=========>....................] - ETA: 1s - loss: 1.3504 - accuracy: 0.5263
566/1563 [=========>....................] - ETA: 1s - loss: 1.3537 - accuracy: 0.5253
604/1563 [==========>...................] - ETA: 1s - loss: 1.3569 - accuracy: 0.5226
642/1563 [===========>..................] - ETA: 1s - loss: 1.3581 - accuracy: 0.5220
679/1563 [============>.................] - ETA: 1s - loss: 1.3571 - accuracy: 0.5219
717/1563 [============>.................] - ETA: 1s - loss: 1.3533 - accuracy: 0.5226
755/1563 [=============>................] - ETA: 1s - loss: 1.3552 - accuracy: 0.5227
793/1563 [==============>...............] - ETA: 1s - loss: 1.3563 - accuracy: 0.5222
831/1563 [==============>...............] - ETA: 0s - loss: 1.3568 - accuracy: 0.5226
869/1563 [===============>..............] - ETA: 0s - loss: 1.3590 - accuracy: 0.5214
907/1563 [================>.............] - ETA: 0s - loss: 1.3576 - accuracy: 0.5222
944/1563 [=================>............] - ETA: 0s - loss: 1.3577 - accuracy: 0.5214
982/1563 [=================>............] - ETA: 0s - loss: 1.3574 - accuracy: 0.5216
1020/1563 [==================>...........] - ETA: 0s - loss: 1.3593 - accuracy: 0.5216
1058/1563 [===================>..........] - ETA: 0s - loss: 1.3585 - accuracy: 0.5219
1096/1563 [====================>.........] - ETA: 0s - loss: 1.3584 - accuracy: 0.5220
1134/1563 [====================>.........] - ETA: 0s - loss: 1.3577 - accuracy: 0.5222
1171/1563 [=====================>........] - ETA: 0s - loss: 1.3565 - accuracy: 0.5226
1209/1563 [======================>.......] - ETA: 0s - loss: 1.3562 - accuracy: 0.5229
1246/1563 [======================>.......] - ETA: 0s - loss: 1.3555 - accuracy: 0.5230
1284/1563 [=======================>......] - ETA: 0s - loss: 1.3547 - accuracy: 0.5233
1322/1563 [========================>.....] - ETA: 0s - loss: 1.3536 - accuracy: 0.5235
1360/1563 [=========================>....] - ETA: 0s - loss: 1.3531 - accuracy: 0.5238
1399/1563 [=========================>....] - ETA: 0s - loss: 1.3528 - accuracy: 0.5238
1437/1563 [==========================>...] - ETA: 0s - loss: 1.3523 - accuracy: 0.5239
1475/1563 [===========================>..] - ETA: 0s - loss: 1.3527 - accuracy: 0.5239
1513/1563 [============================>.] - ETA: 0s - loss: 1.3524 - accuracy: 0.5240
1551/1563 [============================>.] - ETA: 0s - loss: 1.3525 - accuracy: 0.5240
1563/1563 [==============================] - 2s 1ms/step - loss: 1.3524 - accuracy: 0.5238
Epoch 8/10
1/1563 [..............................] - ETA: 2s - loss: 1.2660 - accuracy: 0.5938
39/1563 [..............................] - ETA: 2s - loss: 1.3038 - accuracy: 0.5401
77/1563 [>.............................] - ETA: 1s - loss: 1.2802 - accuracy: 0.5414
115/1563 [=>............................] - ETA: 1s - loss: 1.2991 - accuracy: 0.5321
153/1563 [=>............................] - ETA: 1s - loss: 1.3168 - accuracy: 0.5323
191/1563 [==>...........................] - ETA: 1s - loss: 1.3136 - accuracy: 0.5366
228/1563 [===>..........................] - ETA: 1s - loss: 1.3172 - accuracy: 0.5328
266/1563 [====>.........................] - ETA: 1s - loss: 1.3198 - accuracy: 0.5335
304/1563 [====>.........................] - ETA: 1s - loss: 1.3175 - accuracy: 0.5356
340/1563 [=====>........................] - ETA: 1s - loss: 1.3230 - accuracy: 0.5344
378/1563 [======>.......................] - ETA: 1s - loss: 1.3194 - accuracy: 0.5335
416/1563 [======>.......................] - ETA: 1s - loss: 1.3173 - accuracy: 0.5340
454/1563 [=======>......................] - ETA: 1s - loss: 1.3212 - accuracy: 0.5312
492/1563 [========>.....................] - ETA: 1s - loss: 1.3216 - accuracy: 0.5316
530/1563 [=========>....................] - ETA: 1s - loss: 1.3204 - accuracy: 0.5320
568/1563 [=========>....................] - ETA: 1s - loss: 1.3202 - accuracy: 0.5331
605/1563 [==========>...................] - ETA: 1s - loss: 1.3190 - accuracy: 0.5328
643/1563 [===========>..................] - ETA: 1s - loss: 1.3174 - accuracy: 0.5338
681/1563 [============>.................] - ETA: 1s - loss: 1.3186 - accuracy: 0.5340
719/1563 [============>.................] - ETA: 1s - loss: 1.3197 - accuracy: 0.5339
757/1563 [=============>................] - ETA: 1s - loss: 1.3201 - accuracy: 0.5338
795/1563 [==============>...............] - ETA: 1s - loss: 1.3186 - accuracy: 0.5351
833/1563 [==============>...............] - ETA: 0s - loss: 1.3170 - accuracy: 0.5354
871/1563 [===============>..............] - ETA: 0s - loss: 1.3186 - accuracy: 0.5355
909/1563 [================>.............] - ETA: 0s - loss: 1.3192 - accuracy: 0.5361
947/1563 [=================>............] - ETA: 0s - loss: 1.3186 - accuracy: 0.5372
985/1563 [=================>............] - ETA: 0s - loss: 1.3199 - accuracy: 0.5366
1023/1563 [==================>...........] - ETA: 0s - loss: 1.3191 - accuracy: 0.5371
1061/1563 [===================>..........] - ETA: 0s - loss: 1.3182 - accuracy: 0.5370
1099/1563 [====================>.........] - ETA: 0s - loss: 1.3192 - accuracy: 0.5363
1137/1563 [====================>.........] - ETA: 0s - loss: 1.3176 - accuracy: 0.5367
1175/1563 [=====================>........] - ETA: 0s - loss: 1.3169 - accuracy: 0.5372
1213/1563 [======================>.......] - ETA: 0s - loss: 1.3171 - accuracy: 0.5372
1250/1563 [======================>.......] - ETA: 0s - loss: 1.3162 - accuracy: 0.5376
1288/1563 [=======================>......] - ETA: 0s - loss: 1.3156 - accuracy: 0.5379
1326/1563 [========================>.....] - ETA: 0s - loss: 1.3163 - accuracy: 0.5375
1364/1563 [=========================>....] - ETA: 0s - loss: 1.3152 - accuracy: 0.5381
1402/1563 [=========================>....] - ETA: 0s - loss: 1.3141 - accuracy: 0.5381
1440/1563 [==========================>...] - ETA: 0s - loss: 1.3162 - accuracy: 0.5376
1478/1563 [===========================>..] - ETA: 0s - loss: 1.3155 - accuracy: 0.5373
1516/1563 [============================>.] - ETA: 0s - loss: 1.3157 - accuracy: 0.5370
1554/1563 [============================>.] - ETA: 0s - loss: 1.3161 - accuracy: 0.5367
1563/1563 [==============================] - 2s 1ms/step - loss: 1.3165 - accuracy: 0.5365
Epoch 9/10
1/1563 [..............................] - ETA: 2s - loss: 1.6358 - accuracy: 0.4375
39/1563 [..............................] - ETA: 2s - loss: 1.3514 - accuracy: 0.5192
77/1563 [>.............................] - ETA: 1s - loss: 1.3139 - accuracy: 0.5414
115/1563 [=>............................] - ETA: 1s - loss: 1.3075 - accuracy: 0.5446
153/1563 [=>............................] - ETA: 1s - loss: 1.3046 - accuracy: 0.5468
191/1563 [==>...........................] - ETA: 1s - loss: 1.3061 - accuracy: 0.5448
229/1563 [===>..........................] - ETA: 1s - loss: 1.3023 - accuracy: 0.5461
267/1563 [====>.........................] - ETA: 1s - loss: 1.3049 - accuracy: 0.5440
305/1563 [====>.........................] - ETA: 1s - loss: 1.3022 - accuracy: 0.5443
343/1563 [=====>........................] - ETA: 1s - loss: 1.2986 - accuracy: 0.5452
381/1563 [======>.......................] - ETA: 1s - loss: 1.3013 - accuracy: 0.5431
419/1563 [=======>......................] - ETA: 1s - loss: 1.2989 - accuracy: 0.5450
457/1563 [=======>......................] - ETA: 1s - loss: 1.2994 - accuracy: 0.5452
495/1563 [========>.....................] - ETA: 1s - loss: 1.2987 - accuracy: 0.5446
532/1563 [=========>....................] - ETA: 1s - loss: 1.3000 - accuracy: 0.5439
570/1563 [=========>....................] - ETA: 1s - loss: 1.2973 - accuracy: 0.5440
608/1563 [==========>...................] - ETA: 1s - loss: 1.2949 - accuracy: 0.5448
646/1563 [===========>..................] - ETA: 1s - loss: 1.2954 - accuracy: 0.5451
684/1563 [============>.................] - ETA: 1s - loss: 1.2938 - accuracy: 0.5455
722/1563 [============>.................] - ETA: 1s - loss: 1.2923 - accuracy: 0.5462
760/1563 [=============>................] - ETA: 1s - loss: 1.2930 - accuracy: 0.5463
798/1563 [==============>...............] - ETA: 1s - loss: 1.2942 - accuracy: 0.5461
835/1563 [===============>..............] - ETA: 0s - loss: 1.2913 - accuracy: 0.5472
872/1563 [===============>..............] - ETA: 0s - loss: 1.2916 - accuracy: 0.5468
910/1563 [================>.............] - ETA: 0s - loss: 1.2902 - accuracy: 0.5474
948/1563 [=================>............] - ETA: 0s - loss: 1.2902 - accuracy: 0.5474
986/1563 [=================>............] - ETA: 0s - loss: 1.2891 - accuracy: 0.5471
1024/1563 [==================>...........] - ETA: 0s - loss: 1.2868 - accuracy: 0.5481
1062/1563 [===================>..........] - ETA: 0s - loss: 1.2861 - accuracy: 0.5489
1100/1563 [====================>.........] - ETA: 0s - loss: 1.2861 - accuracy: 0.5490
1138/1563 [====================>.........] - ETA: 0s - loss: 1.2857 - accuracy: 0.5490
1175/1563 [=====================>........] - ETA: 0s - loss: 1.2859 - accuracy: 0.5485
1213/1563 [======================>.......] - ETA: 0s - loss: 1.2856 - accuracy: 0.5484
1251/1563 [=======================>......] - ETA: 0s - loss: 1.2846 - accuracy: 0.5487
1289/1563 [=======================>......] - ETA: 0s - loss: 1.2862 - accuracy: 0.5486
1327/1563 [========================>.....] - ETA: 0s - loss: 1.2856 - accuracy: 0.5491
1365/1563 [=========================>....] - ETA: 0s - loss: 1.2852 - accuracy: 0.5488
1403/1563 [=========================>....] - ETA: 0s - loss: 1.2862 - accuracy: 0.5483
1441/1563 [==========================>...] - ETA: 0s - loss: 1.2850 - accuracy: 0.5488
1479/1563 [===========================>..] - ETA: 0s - loss: 1.2845 - accuracy: 0.5489
1517/1563 [============================>.] - ETA: 0s - loss: 1.2837 - accuracy: 0.5487
1555/1563 [============================>.] - ETA: 0s - loss: 1.2841 - accuracy: 0.5483
1563/1563 [==============================] - 2s 1ms/step - loss: 1.2840 - accuracy: 0.5483
Epoch 10/10
1/1563 [..............................] - ETA: 2s - loss: 1.2325 - accuracy: 0.5312
39/1563 [..............................] - ETA: 2s - loss: 1.2643 - accuracy: 0.5593
77/1563 [>.............................] - ETA: 1s - loss: 1.2671 - accuracy: 0.5556
115/1563 [=>............................] - ETA: 1s - loss: 1.2489 - accuracy: 0.5554
153/1563 [=>............................] - ETA: 1s - loss: 1.2436 - accuracy: 0.5580
191/1563 [==>...........................] - ETA: 1s - loss: 1.2600 - accuracy: 0.5525
229/1563 [===>..........................] - ETA: 1s - loss: 1.2562 - accuracy: 0.5535
267/1563 [====>.........................] - ETA: 1s - loss: 1.2552 - accuracy: 0.5563
304/1563 [====>.........................] - ETA: 1s - loss: 1.2624 - accuracy: 0.5552
341/1563 [=====>........................] - ETA: 1s - loss: 1.2635 - accuracy: 0.5540
379/1563 [======>.......................] - ETA: 1s - loss: 1.2586 - accuracy: 0.5557
416/1563 [======>.......................] - ETA: 1s - loss: 1.2628 - accuracy: 0.5540
454/1563 [=======>......................] - ETA: 1s - loss: 1.2661 - accuracy: 0.5541
492/1563 [========>.....................] - ETA: 1s - loss: 1.2654 - accuracy: 0.5561
530/1563 [=========>....................] - ETA: 1s - loss: 1.2671 - accuracy: 0.5547
568/1563 [=========>....................] - ETA: 1s - loss: 1.2627 - accuracy: 0.5553
606/1563 [==========>...................] - ETA: 1s - loss: 1.2628 - accuracy: 0.5551
643/1563 [===========>..................] - ETA: 1s - loss: 1.2646 - accuracy: 0.5554
681/1563 [============>.................] - ETA: 1s - loss: 1.2623 - accuracy: 0.5556
719/1563 [============>.................] - ETA: 1s - loss: 1.2614 - accuracy: 0.5546
757/1563 [=============>................] - ETA: 1s - loss: 1.2606 - accuracy: 0.5553
795/1563 [==============>...............] - ETA: 1s - loss: 1.2602 - accuracy: 0.5553
833/1563 [==============>...............] - ETA: 0s - loss: 1.2611 - accuracy: 0.5544
871/1563 [===============>..............] - ETA: 0s - loss: 1.2614 - accuracy: 0.5543
909/1563 [================>.............] - ETA: 0s - loss: 1.2622 - accuracy: 0.5545
947/1563 [=================>............] - ETA: 0s - loss: 1.2628 - accuracy: 0.5551
985/1563 [=================>............] - ETA: 0s - loss: 1.2618 - accuracy: 0.5563
1023/1563 [==================>...........] - ETA: 0s - loss: 1.2602 - accuracy: 0.5572
1060/1563 [===================>..........] - ETA: 0s - loss: 1.2599 - accuracy: 0.5574
1098/1563 [====================>.........] - ETA: 0s - loss: 1.2588 - accuracy: 0.5582
1136/1563 [====================>.........] - ETA: 0s - loss: 1.2571 - accuracy: 0.5588
1174/1563 [=====================>........] - ETA: 0s - loss: 1.2557 - accuracy: 0.5590
1211/1563 [======================>.......] - ETA: 0s - loss: 1.2554 - accuracy: 0.5588
1249/1563 [======================>.......] - ETA: 0s - loss: 1.2544 - accuracy: 0.5594
1287/1563 [=======================>......] - ETA: 0s - loss: 1.2548 - accuracy: 0.5593
1325/1563 [========================>.....] - ETA: 0s - loss: 1.2544 - accuracy: 0.5591
1363/1563 [=========================>....] - ETA: 0s - loss: 1.2559 - accuracy: 0.5585
1401/1563 [=========================>....] - ETA: 0s - loss: 1.2561 - accuracy: 0.5585
1439/1563 [==========================>...] - ETA: 0s - loss: 1.2557 - accuracy: 0.5582
1477/1563 [===========================>..] - ETA: 0s - loss: 1.2555 - accuracy: 0.5582
1515/1563 [============================>.] - ETA: 0s - loss: 1.2557 - accuracy: 0.5580
1553/1563 [============================>.] - ETA: 0s - loss: 1.2538 - accuracy: 0.5586
1563/1563 [==============================] - 2s 1ms/step - loss: 1.2531 - accuracy: 0.5588
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
flatten (Flatten) (None, 3072) 0
dense (Dense) (None, 3000) 9219000
dense_1 (Dense) (None, 1000) 3001000
dense_2 (Dense) (None, 10) 10010
=================================================================
Total params: 12,230,010
Trainable params: 12,230,010
Non-trainable params: 0
_________________________________________________________________
None
wall time seconds (without loading test data set)= 37.25028657913208
hydro05
[SSH] completed
[SSH] exit-status: 0
[workspace] $ /bin/sh -xe /tmp/jenkins4177595130164554911.sh
+ scp 'HYDRO_REMOTE:~svchydrojenkins/python/tensorflow_time.txt' /var/lib/jenkins/jobs/tensorflow/workspace
Recording plot data
Saving plot series data from: /var/lib/jenkins/jobs/tensorflow/workspace/tensorflow_time.txt
Finished: SUCCESS
![[Jenkins]](/static/755cd4af/images/svgs/logo.svg){kind=logo}
