Started by timer
Running as SYSTEM
Building in workspace /var/lib/jenkins/jobs/tensorflow/workspace
[SSH] script:
module load anaconda3_gpu
module load cudnn
cd python
rm -f tensorflow_test.txt
srun --nodes=1 --ntasks-per-node=1 --mem=16g \
--time=00:05:00 \
--partition=a100rome \
--account=bbmb-hydro \
--gpus-per-node=1 \
--gpu-bind=closest \
python3 cifar10gpu.py | tee tensorflow_test.txt
YVALUE=`grep "wall time seconds" tensorflow_test.txt | cut -d'=' -f2`
echo YVALUE=$YVALUE > tensorflow_time.txt
[SSH] executing...
srun: job 84649 queued and waiting for resources
srun: job 84649 has been allocated resources
2024-10-12 07:43:08.046640: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-10-12 07:43:34.117327: E tensorflow/stream_executor/cuda/cuda_blas.cc:2981] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-10-12 07:44:10.165651: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /sw/spack/hydrogpu-2022-06/apps/cudnn/8.4.0.27-11.6-gcc-11.3.0-qbhsfav/lib:/sw/spack/hydrogpu-2022-06/apps/cuda/11.7.0-gcc-11.3.0-3ysno6b/lib64:/sw/spack/hydrogpu-2022-06/apps/openmpi/4.1.4-gcc-11.3.0-i6koipj/lib:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib64:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib
2024-10-12 07:44:10.166226: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /sw/spack/hydrogpu-2022-06/apps/cudnn/8.4.0.27-11.6-gcc-11.3.0-qbhsfav/lib:/sw/spack/hydrogpu-2022-06/apps/cuda/11.7.0-gcc-11.3.0-3ysno6b/lib64:/sw/spack/hydrogpu-2022-06/apps/openmpi/4.1.4-gcc-11.3.0-i6koipj/lib:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib64:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib
2024-10-12 07:44:10.166235: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
2024-10-12 07:45:07.534376: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-10-12 07:45:10.456337: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1616] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 79030 MB memory: -> device: 0, name: NVIDIA A100 80GB PCIe, pci bus id: 0000:21:00.0, compute capability: 8.0
(50000, 32, 32, 3) (10000, 32, 32, 3)
Epoch 1/10
2024-10-12 07:45:19.415345: I tensorflow/stream_executor/cuda/cuda_blas.cc:1614] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
1/1563 [..............................] - ETA: 1:51:26 - loss: 2.3124 - accuracy: 0.1250
37/1563 [..............................] - ETA: 2s - loss: 2.2395 - accuracy: 0.1715
75/1563 [>.............................] - ETA: 2s - loss: 2.1598 - accuracy: 0.2025
113/1563 [=>............................] - ETA: 1s - loss: 2.1118 - accuracy: 0.2270
151/1563 [=>............................] - ETA: 1s - loss: 2.0792 - accuracy: 0.2432
189/1563 [==>...........................] - ETA: 1s - loss: 2.0502 - accuracy: 0.2561
227/1563 [===>..........................] - ETA: 1s - loss: 2.0255 - accuracy: 0.2642
265/1563 [====>.........................] - ETA: 1s - loss: 2.0097 - accuracy: 0.2691
303/1563 [====>.........................] - ETA: 1s - loss: 1.9946 - accuracy: 0.2745
341/1563 [=====>........................] - ETA: 1s - loss: 1.9788 - accuracy: 0.2810
379/1563 [======>.......................] - ETA: 1s - loss: 1.9701 - accuracy: 0.2864
417/1563 [=======>......................] - ETA: 1s - loss: 1.9579 - accuracy: 0.2922
455/1563 [=======>......................] - ETA: 1s - loss: 1.9482 - accuracy: 0.2966
493/1563 [========>.....................] - ETA: 1s - loss: 1.9374 - accuracy: 0.3009
531/1563 [=========>....................] - ETA: 1s - loss: 1.9319 - accuracy: 0.3040
569/1563 [=========>....................] - ETA: 1s - loss: 1.9254 - accuracy: 0.3072
607/1563 [==========>...................] - ETA: 1s - loss: 1.9184 - accuracy: 0.3103
646/1563 [===========>..................] - ETA: 1s - loss: 1.9132 - accuracy: 0.3136
684/1563 [============>.................] - ETA: 1s - loss: 1.9056 - accuracy: 0.3167
722/1563 [============>.................] - ETA: 1s - loss: 1.8998 - accuracy: 0.3195
760/1563 [=============>................] - ETA: 1s - loss: 1.8945 - accuracy: 0.3215
798/1563 [==============>...............] - ETA: 1s - loss: 1.8895 - accuracy: 0.3230
837/1563 [===============>..............] - ETA: 0s - loss: 1.8831 - accuracy: 0.3251
875/1563 [===============>..............] - ETA: 0s - loss: 1.8780 - accuracy: 0.3278
913/1563 [================>.............] - ETA: 0s - loss: 1.8744 - accuracy: 0.3298
951/1563 [=================>............] - ETA: 0s - loss: 1.8709 - accuracy: 0.3312
989/1563 [=================>............] - ETA: 0s - loss: 1.8678 - accuracy: 0.3323
1027/1563 [==================>...........] - ETA: 0s - loss: 1.8619 - accuracy: 0.3351
1065/1563 [===================>..........] - ETA: 0s - loss: 1.8573 - accuracy: 0.3374
1103/1563 [====================>.........] - ETA: 0s - loss: 1.8524 - accuracy: 0.3388
1141/1563 [====================>.........] - ETA: 0s - loss: 1.8494 - accuracy: 0.3397
1180/1563 [=====================>........] - ETA: 0s - loss: 1.8466 - accuracy: 0.3407
1218/1563 [======================>.......] - ETA: 0s - loss: 1.8424 - accuracy: 0.3426
1257/1563 [=======================>......] - ETA: 0s - loss: 1.8373 - accuracy: 0.3447
1295/1563 [=======================>......] - ETA: 0s - loss: 1.8319 - accuracy: 0.3471
1334/1563 [========================>.....] - ETA: 0s - loss: 1.8287 - accuracy: 0.3485
1372/1563 [=========================>....] - ETA: 0s - loss: 1.8264 - accuracy: 0.3496
1411/1563 [==========================>...] - ETA: 0s - loss: 1.8236 - accuracy: 0.3509
1449/1563 [==========================>...] - ETA: 0s - loss: 1.8204 - accuracy: 0.3520
1488/1563 [===========================>..] - ETA: 0s - loss: 1.8171 - accuracy: 0.3531
1527/1563 [============================>.] - ETA: 0s - loss: 1.8136 - accuracy: 0.3542
1563/1563 [==============================] - 6s 1ms/step - loss: 1.8106 - accuracy: 0.3556
Epoch 2/10
1/1563 [..............................] - ETA: 2s - loss: 1.6469 - accuracy: 0.4688
39/1563 [..............................] - ETA: 2s - loss: 1.6997 - accuracy: 0.3982
77/1563 [>.............................] - ETA: 1s - loss: 1.6920 - accuracy: 0.4042
115/1563 [=>............................] - ETA: 1s - loss: 1.6817 - accuracy: 0.4035
153/1563 [=>............................] - ETA: 1s - loss: 1.6854 - accuracy: 0.4046
191/1563 [==>...........................] - ETA: 1s - loss: 1.6756 - accuracy: 0.4090
229/1563 [===>..........................] - ETA: 1s - loss: 1.6758 - accuracy: 0.4084
268/1563 [====>.........................] - ETA: 1s - loss: 1.6738 - accuracy: 0.4072
306/1563 [====>.........................] - ETA: 1s - loss: 1.6686 - accuracy: 0.4117
345/1563 [=====>........................] - ETA: 1s - loss: 1.6662 - accuracy: 0.4125
384/1563 [======>.......................] - ETA: 1s - loss: 1.6693 - accuracy: 0.4101
422/1563 [=======>......................] - ETA: 1s - loss: 1.6637 - accuracy: 0.4108
461/1563 [=======>......................] - ETA: 1s - loss: 1.6613 - accuracy: 0.4124
499/1563 [========>.....................] - ETA: 1s - loss: 1.6556 - accuracy: 0.4144
537/1563 [=========>....................] - ETA: 1s - loss: 1.6519 - accuracy: 0.4164
575/1563 [==========>...................] - ETA: 1s - loss: 1.6515 - accuracy: 0.4153
614/1563 [==========>...................] - ETA: 1s - loss: 1.6511 - accuracy: 0.4159
653/1563 [===========>..................] - ETA: 1s - loss: 1.6511 - accuracy: 0.4152
692/1563 [============>.................] - ETA: 1s - loss: 1.6496 - accuracy: 0.4174
730/1563 [=============>................] - ETA: 1s - loss: 1.6492 - accuracy: 0.4172
768/1563 [=============>................] - ETA: 1s - loss: 1.6470 - accuracy: 0.4183
806/1563 [==============>...............] - ETA: 0s - loss: 1.6470 - accuracy: 0.4187
844/1563 [===============>..............] - ETA: 0s - loss: 1.6440 - accuracy: 0.4198
882/1563 [===============>..............] - ETA: 0s - loss: 1.6427 - accuracy: 0.4200
920/1563 [================>.............] - ETA: 0s - loss: 1.6384 - accuracy: 0.4216
958/1563 [=================>............] - ETA: 0s - loss: 1.6387 - accuracy: 0.4210
996/1563 [==================>...........] - ETA: 0s - loss: 1.6371 - accuracy: 0.4210
1034/1563 [==================>...........] - ETA: 0s - loss: 1.6361 - accuracy: 0.4221
1072/1563 [===================>..........] - ETA: 0s - loss: 1.6350 - accuracy: 0.4225
1110/1563 [====================>.........] - ETA: 0s - loss: 1.6321 - accuracy: 0.4235
1149/1563 [=====================>........] - ETA: 0s - loss: 1.6312 - accuracy: 0.4238
1188/1563 [=====================>........] - ETA: 0s - loss: 1.6274 - accuracy: 0.4249
1226/1563 [======================>.......] - ETA: 0s - loss: 1.6271 - accuracy: 0.4252
1264/1563 [=======================>......] - ETA: 0s - loss: 1.6274 - accuracy: 0.4251
1302/1563 [=======================>......] - ETA: 0s - loss: 1.6272 - accuracy: 0.4249
1340/1563 [========================>.....] - ETA: 0s - loss: 1.6256 - accuracy: 0.4257
1378/1563 [=========================>....] - ETA: 0s - loss: 1.6254 - accuracy: 0.4261
1416/1563 [==========================>...] - ETA: 0s - loss: 1.6241 - accuracy: 0.4262
1454/1563 [==========================>...] - ETA: 0s - loss: 1.6230 - accuracy: 0.4266
1492/1563 [===========================>..] - ETA: 0s - loss: 1.6229 - accuracy: 0.4267
1530/1563 [============================>.] - ETA: 0s - loss: 1.6226 - accuracy: 0.4267
1563/1563 [==============================] - 2s 1ms/step - loss: 1.6213 - accuracy: 0.4269
Epoch 3/10
1/1563 [..............................] - ETA: 2s - loss: 1.3695 - accuracy: 0.5000
39/1563 [..............................] - ETA: 2s - loss: 1.5389 - accuracy: 0.4559
78/1563 [>.............................] - ETA: 1s - loss: 1.5578 - accuracy: 0.4547
116/1563 [=>............................] - ETA: 1s - loss: 1.5639 - accuracy: 0.4494
154/1563 [=>............................] - ETA: 1s - loss: 1.5660 - accuracy: 0.4454
193/1563 [==>...........................] - ETA: 1s - loss: 1.5569 - accuracy: 0.4519
232/1563 [===>..........................] - ETA: 1s - loss: 1.5576 - accuracy: 0.4500
270/1563 [====>.........................] - ETA: 1s - loss: 1.5536 - accuracy: 0.4505
308/1563 [====>.........................] - ETA: 1s - loss: 1.5541 - accuracy: 0.4491
346/1563 [=====>........................] - ETA: 1s - loss: 1.5502 - accuracy: 0.4515
384/1563 [======>.......................] - ETA: 1s - loss: 1.5478 - accuracy: 0.4514
422/1563 [=======>......................] - ETA: 1s - loss: 1.5482 - accuracy: 0.4516
461/1563 [=======>......................] - ETA: 1s - loss: 1.5481 - accuracy: 0.4521
499/1563 [========>.....................] - ETA: 1s - loss: 1.5467 - accuracy: 0.4543
538/1563 [=========>....................] - ETA: 1s - loss: 1.5459 - accuracy: 0.4546
576/1563 [==========>...................] - ETA: 1s - loss: 1.5436 - accuracy: 0.4555
614/1563 [==========>...................] - ETA: 1s - loss: 1.5428 - accuracy: 0.4545
652/1563 [===========>..................] - ETA: 1s - loss: 1.5452 - accuracy: 0.4535
690/1563 [============>.................] - ETA: 1s - loss: 1.5437 - accuracy: 0.4542
728/1563 [============>.................] - ETA: 1s - loss: 1.5451 - accuracy: 0.4543
766/1563 [=============>................] - ETA: 1s - loss: 1.5438 - accuracy: 0.4552
805/1563 [==============>...............] - ETA: 1s - loss: 1.5440 - accuracy: 0.4555
844/1563 [===============>..............] - ETA: 0s - loss: 1.5412 - accuracy: 0.4570
883/1563 [===============>..............] - ETA: 0s - loss: 1.5400 - accuracy: 0.4580
921/1563 [================>.............] - ETA: 0s - loss: 1.5402 - accuracy: 0.4578
959/1563 [=================>............] - ETA: 0s - loss: 1.5382 - accuracy: 0.4583
997/1563 [==================>...........] - ETA: 0s - loss: 1.5378 - accuracy: 0.4588
1035/1563 [==================>...........] - ETA: 0s - loss: 1.5372 - accuracy: 0.4591
1073/1563 [===================>..........] - ETA: 0s - loss: 1.5378 - accuracy: 0.4588
1111/1563 [====================>.........] - ETA: 0s - loss: 1.5391 - accuracy: 0.4583
1150/1563 [=====================>........] - ETA: 0s - loss: 1.5388 - accuracy: 0.4585
1189/1563 [=====================>........] - ETA: 0s - loss: 1.5374 - accuracy: 0.4590
1228/1563 [======================>.......] - ETA: 0s - loss: 1.5363 - accuracy: 0.4594
1266/1563 [=======================>......] - ETA: 0s - loss: 1.5359 - accuracy: 0.4598
1304/1563 [========================>.....] - ETA: 0s - loss: 1.5361 - accuracy: 0.4597
1342/1563 [========================>.....] - ETA: 0s - loss: 1.5363 - accuracy: 0.4598
1380/1563 [=========================>....] - ETA: 0s - loss: 1.5360 - accuracy: 0.4599
1419/1563 [==========================>...] - ETA: 0s - loss: 1.5359 - accuracy: 0.4597
1458/1563 [==========================>...] - ETA: 0s - loss: 1.5367 - accuracy: 0.4591
1496/1563 [===========================>..] - ETA: 0s - loss: 1.5365 - accuracy: 0.4590
1534/1563 [============================>.] - ETA: 0s - loss: 1.5385 - accuracy: 0.4586
1563/1563 [==============================] - 2s 1ms/step - loss: 1.5384 - accuracy: 0.4585
Epoch 4/10
1/1563 [..............................] - ETA: 2s - loss: 1.4855 - accuracy: 0.4375
39/1563 [..............................] - ETA: 2s - loss: 1.4662 - accuracy: 0.4840
77/1563 [>.............................] - ETA: 1s - loss: 1.4795 - accuracy: 0.4821
115/1563 [=>............................] - ETA: 1s - loss: 1.4707 - accuracy: 0.4848
154/1563 [=>............................] - ETA: 1s - loss: 1.4679 - accuracy: 0.4834
193/1563 [==>...........................] - ETA: 1s - loss: 1.4803 - accuracy: 0.4785
232/1563 [===>..........................] - ETA: 1s - loss: 1.4857 - accuracy: 0.4752
271/1563 [====>.........................] - ETA: 1s - loss: 1.4855 - accuracy: 0.4732
310/1563 [====>.........................] - ETA: 1s - loss: 1.4873 - accuracy: 0.4745
349/1563 [=====>........................] - ETA: 1s - loss: 1.4869 - accuracy: 0.4765
387/1563 [======>.......................] - ETA: 1s - loss: 1.4857 - accuracy: 0.4788
425/1563 [=======>......................] - ETA: 1s - loss: 1.4813 - accuracy: 0.4795
464/1563 [=======>......................] - ETA: 1s - loss: 1.4818 - accuracy: 0.4803
502/1563 [========>.....................] - ETA: 1s - loss: 1.4836 - accuracy: 0.4800
541/1563 [=========>....................] - ETA: 1s - loss: 1.4862 - accuracy: 0.4787
579/1563 [==========>...................] - ETA: 1s - loss: 1.4853 - accuracy: 0.4792
617/1563 [==========>...................] - ETA: 1s - loss: 1.4854 - accuracy: 0.4796
655/1563 [===========>..................] - ETA: 1s - loss: 1.4854 - accuracy: 0.4789
693/1563 [============>.................] - ETA: 1s - loss: 1.4862 - accuracy: 0.4792
731/1563 [=============>................] - ETA: 1s - loss: 1.4867 - accuracy: 0.4789
769/1563 [=============>................] - ETA: 1s - loss: 1.4871 - accuracy: 0.4790
808/1563 [==============>...............] - ETA: 0s - loss: 1.4874 - accuracy: 0.4788
847/1563 [===============>..............] - ETA: 0s - loss: 1.4878 - accuracy: 0.4782
885/1563 [===============>..............] - ETA: 0s - loss: 1.4849 - accuracy: 0.4797
923/1563 [================>.............] - ETA: 0s - loss: 1.4845 - accuracy: 0.4801
961/1563 [=================>............] - ETA: 0s - loss: 1.4831 - accuracy: 0.4805
999/1563 [==================>...........] - ETA: 0s - loss: 1.4840 - accuracy: 0.4797
1037/1563 [==================>...........] - ETA: 0s - loss: 1.4835 - accuracy: 0.4793
1076/1563 [===================>..........] - ETA: 0s - loss: 1.4835 - accuracy: 0.4788
1114/1563 [====================>.........] - ETA: 0s - loss: 1.4834 - accuracy: 0.4786
1153/1563 [=====================>........] - ETA: 0s - loss: 1.4826 - accuracy: 0.4787
1191/1563 [=====================>........] - ETA: 0s - loss: 1.4827 - accuracy: 0.4782
1229/1563 [======================>.......] - ETA: 0s - loss: 1.4822 - accuracy: 0.4786
1267/1563 [=======================>......] - ETA: 0s - loss: 1.4821 - accuracy: 0.4788
1305/1563 [========================>.....] - ETA: 0s - loss: 1.4826 - accuracy: 0.4786
1343/1563 [========================>.....] - ETA: 0s - loss: 1.4826 - accuracy: 0.4785
1382/1563 [=========================>....] - ETA: 0s - loss: 1.4815 - accuracy: 0.4789
1421/1563 [==========================>...] - ETA: 0s - loss: 1.4806 - accuracy: 0.4788
1460/1563 [===========================>..] - ETA: 0s - loss: 1.4809 - accuracy: 0.4789
1499/1563 [===========================>..] - ETA: 0s - loss: 1.4800 - accuracy: 0.4788
1538/1563 [============================>.] - ETA: 0s - loss: 1.4799 - accuracy: 0.4788
1563/1563 [==============================] - 2s 1ms/step - loss: 1.4796 - accuracy: 0.4789
Epoch 5/10
1/1563 [..............................] - ETA: 2s - loss: 1.5506 - accuracy: 0.4375
39/1563 [..............................] - ETA: 2s - loss: 1.4903 - accuracy: 0.4607
78/1563 [>.............................] - ETA: 1s - loss: 1.4711 - accuracy: 0.4800
117/1563 [=>............................] - ETA: 1s - loss: 1.4557 - accuracy: 0.4912
155/1563 [=>............................] - ETA: 1s - loss: 1.4451 - accuracy: 0.4962
194/1563 [==>...........................] - ETA: 1s - loss: 1.4357 - accuracy: 0.5006
232/1563 [===>..........................] - ETA: 1s - loss: 1.4297 - accuracy: 0.5008
271/1563 [====>.........................] - ETA: 1s - loss: 1.4302 - accuracy: 0.4985
309/1563 [====>.........................] - ETA: 1s - loss: 1.4373 - accuracy: 0.4964
347/1563 [=====>........................] - ETA: 1s - loss: 1.4343 - accuracy: 0.4959
385/1563 [======>.......................] - ETA: 1s - loss: 1.4339 - accuracy: 0.4974
424/1563 [=======>......................] - ETA: 1s - loss: 1.4365 - accuracy: 0.4946
463/1563 [=======>......................] - ETA: 1s - loss: 1.4320 - accuracy: 0.4968
501/1563 [========>.....................] - ETA: 1s - loss: 1.4378 - accuracy: 0.4935
539/1563 [=========>....................] - ETA: 1s - loss: 1.4379 - accuracy: 0.4928
577/1563 [==========>...................] - ETA: 1s - loss: 1.4355 - accuracy: 0.4945
616/1563 [==========>...................] - ETA: 1s - loss: 1.4372 - accuracy: 0.4930
654/1563 [===========>..................] - ETA: 1s - loss: 1.4337 - accuracy: 0.4945
693/1563 [============>.................] - ETA: 1s - loss: 1.4322 - accuracy: 0.4946
732/1563 [=============>................] - ETA: 1s - loss: 1.4335 - accuracy: 0.4941
770/1563 [=============>................] - ETA: 1s - loss: 1.4333 - accuracy: 0.4942
809/1563 [==============>...............] - ETA: 0s - loss: 1.4315 - accuracy: 0.4949
847/1563 [===============>..............] - ETA: 0s - loss: 1.4329 - accuracy: 0.4943
885/1563 [===============>..............] - ETA: 0s - loss: 1.4327 - accuracy: 0.4946
923/1563 [================>.............] - ETA: 0s - loss: 1.4324 - accuracy: 0.4948
961/1563 [=================>............] - ETA: 0s - loss: 1.4312 - accuracy: 0.4959
999/1563 [==================>...........] - ETA: 0s - loss: 1.4303 - accuracy: 0.4966
1037/1563 [==================>...........] - ETA: 0s - loss: 1.4298 - accuracy: 0.4966
1075/1563 [===================>..........] - ETA: 0s - loss: 1.4311 - accuracy: 0.4961
1114/1563 [====================>.........] - ETA: 0s - loss: 1.4332 - accuracy: 0.4952
1152/1563 [=====================>........] - ETA: 0s - loss: 1.4339 - accuracy: 0.4951
1190/1563 [=====================>........] - ETA: 0s - loss: 1.4346 - accuracy: 0.4949
1228/1563 [======================>.......] - ETA: 0s - loss: 1.4333 - accuracy: 0.4957
1266/1563 [=======================>......] - ETA: 0s - loss: 1.4343 - accuracy: 0.4953
1304/1563 [========================>.....] - ETA: 0s - loss: 1.4339 - accuracy: 0.4959
1342/1563 [========================>.....] - ETA: 0s - loss: 1.4319 - accuracy: 0.4965
1380/1563 [=========================>....] - ETA: 0s - loss: 1.4313 - accuracy: 0.4969
1418/1563 [==========================>...] - ETA: 0s - loss: 1.4317 - accuracy: 0.4962
1457/1563 [==========================>...] - ETA: 0s - loss: 1.4311 - accuracy: 0.4963
1496/1563 [===========================>..] - ETA: 0s - loss: 1.4305 - accuracy: 0.4967
1534/1563 [============================>.] - ETA: 0s - loss: 1.4309 - accuracy: 0.4966
1563/1563 [==============================] - 2s 1ms/step - loss: 1.4313 - accuracy: 0.4964
Epoch 6/10
1/1563 [..............................] - ETA: 2s - loss: 0.9298 - accuracy: 0.6875
40/1563 [..............................] - ETA: 2s - loss: 1.3821 - accuracy: 0.5172
78/1563 [>.............................] - ETA: 1s - loss: 1.3752 - accuracy: 0.5152
117/1563 [=>............................] - ETA: 1s - loss: 1.3701 - accuracy: 0.5256
155/1563 [=>............................] - ETA: 1s - loss: 1.3733 - accuracy: 0.5220
193/1563 [==>...........................] - ETA: 1s - loss: 1.3725 - accuracy: 0.5209
232/1563 [===>..........................] - ETA: 1s - loss: 1.3767 - accuracy: 0.5195
271/1563 [====>.........................] - ETA: 1s - loss: 1.3743 - accuracy: 0.5198
310/1563 [====>.........................] - ETA: 1s - loss: 1.3783 - accuracy: 0.5172
348/1563 [=====>........................] - ETA: 1s - loss: 1.3755 - accuracy: 0.5189
386/1563 [======>.......................] - ETA: 1s - loss: 1.3725 - accuracy: 0.5195
424/1563 [=======>......................] - ETA: 1s - loss: 1.3745 - accuracy: 0.5193
462/1563 [=======>......................] - ETA: 1s - loss: 1.3758 - accuracy: 0.5189
501/1563 [========>.....................] - ETA: 1s - loss: 1.3751 - accuracy: 0.5190
540/1563 [=========>....................] - ETA: 1s - loss: 1.3785 - accuracy: 0.5174
579/1563 [==========>...................] - ETA: 1s - loss: 1.3782 - accuracy: 0.5179
617/1563 [==========>...................] - ETA: 1s - loss: 1.3795 - accuracy: 0.5167
655/1563 [===========>..................] - ETA: 1s - loss: 1.3796 - accuracy: 0.5169
694/1563 [============>.................] - ETA: 1s - loss: 1.3833 - accuracy: 0.5164
732/1563 [=============>................] - ETA: 1s - loss: 1.3824 - accuracy: 0.5166
771/1563 [=============>................] - ETA: 1s - loss: 1.3848 - accuracy: 0.5160
810/1563 [==============>...............] - ETA: 0s - loss: 1.3848 - accuracy: 0.5155
849/1563 [===============>..............] - ETA: 0s - loss: 1.3850 - accuracy: 0.5155
888/1563 [================>.............] - ETA: 0s - loss: 1.3840 - accuracy: 0.5162
926/1563 [================>.............] - ETA: 0s - loss: 1.3842 - accuracy: 0.5159
964/1563 [=================>............] - ETA: 0s - loss: 1.3852 - accuracy: 0.5158
1002/1563 [==================>...........] - ETA: 0s - loss: 1.3856 - accuracy: 0.5157
1040/1563 [==================>...........] - ETA: 0s - loss: 1.3860 - accuracy: 0.5155
1078/1563 [===================>..........] - ETA: 0s - loss: 1.3867 - accuracy: 0.5152
1117/1563 [====================>.........] - ETA: 0s - loss: 1.3890 - accuracy: 0.5145
1156/1563 [=====================>........] - ETA: 0s - loss: 1.3898 - accuracy: 0.5141
1194/1563 [=====================>........] - ETA: 0s - loss: 1.3905 - accuracy: 0.5137
1232/1563 [======================>.......] - ETA: 0s - loss: 1.3905 - accuracy: 0.5134
1270/1563 [=======================>......] - ETA: 0s - loss: 1.3892 - accuracy: 0.5132
1308/1563 [========================>.....] - ETA: 0s - loss: 1.3896 - accuracy: 0.5132
1346/1563 [========================>.....] - ETA: 0s - loss: 1.3901 - accuracy: 0.5133
1385/1563 [=========================>....] - ETA: 0s - loss: 1.3900 - accuracy: 0.5139
1424/1563 [==========================>...] - ETA: 0s - loss: 1.3893 - accuracy: 0.5141
1462/1563 [===========================>..] - ETA: 0s - loss: 1.3896 - accuracy: 0.5140
1501/1563 [===========================>..] - ETA: 0s - loss: 1.3892 - accuracy: 0.5143
1539/1563 [============================>.] - ETA: 0s - loss: 1.3888 - accuracy: 0.5144
1563/1563 [==============================] - 2s 1ms/step - loss: 1.3888 - accuracy: 0.5139
Epoch 7/10
1/1563 [..............................] - ETA: 2s - loss: 1.5200 - accuracy: 0.4375
39/1563 [..............................] - ETA: 2s - loss: 1.3828 - accuracy: 0.5168
77/1563 [>.............................] - ETA: 1s - loss: 1.3695 - accuracy: 0.5191
115/1563 [=>............................] - ETA: 1s - loss: 1.3590 - accuracy: 0.5234
154/1563 [=>............................] - ETA: 1s - loss: 1.3603 - accuracy: 0.5229
193/1563 [==>...........................] - ETA: 1s - loss: 1.3577 - accuracy: 0.5261
231/1563 [===>..........................] - ETA: 1s - loss: 1.3558 - accuracy: 0.5267
269/1563 [====>.........................] - ETA: 1s - loss: 1.3555 - accuracy: 0.5272
307/1563 [====>.........................] - ETA: 1s - loss: 1.3536 - accuracy: 0.5293
346/1563 [=====>........................] - ETA: 1s - loss: 1.3546 - accuracy: 0.5269
384/1563 [======>.......................] - ETA: 1s - loss: 1.3519 - accuracy: 0.5269
422/1563 [=======>......................] - ETA: 1s - loss: 1.3511 - accuracy: 0.5266
461/1563 [=======>......................] - ETA: 1s - loss: 1.3511 - accuracy: 0.5275
500/1563 [========>.....................] - ETA: 1s - loss: 1.3525 - accuracy: 0.5264
538/1563 [=========>....................] - ETA: 1s - loss: 1.3496 - accuracy: 0.5281
576/1563 [==========>...................] - ETA: 1s - loss: 1.3506 - accuracy: 0.5279
614/1563 [==========>...................] - ETA: 1s - loss: 1.3521 - accuracy: 0.5267
652/1563 [===========>..................] - ETA: 1s - loss: 1.3506 - accuracy: 0.5269
690/1563 [============>.................] - ETA: 1s - loss: 1.3517 - accuracy: 0.5269
728/1563 [============>.................] - ETA: 1s - loss: 1.3499 - accuracy: 0.5276
767/1563 [=============>................] - ETA: 1s - loss: 1.3499 - accuracy: 0.5262
805/1563 [==============>...............] - ETA: 0s - loss: 1.3506 - accuracy: 0.5264
844/1563 [===============>..............] - ETA: 0s - loss: 1.3494 - accuracy: 0.5268
882/1563 [===============>..............] - ETA: 0s - loss: 1.3487 - accuracy: 0.5265
920/1563 [================>.............] - ETA: 0s - loss: 1.3481 - accuracy: 0.5268
958/1563 [=================>............] - ETA: 0s - loss: 1.3493 - accuracy: 0.5270
996/1563 [==================>...........] - ETA: 0s - loss: 1.3490 - accuracy: 0.5265
1035/1563 [==================>...........] - ETA: 0s - loss: 1.3481 - accuracy: 0.5271
1074/1563 [===================>..........] - ETA: 0s - loss: 1.3487 - accuracy: 0.5264
1113/1563 [====================>.........] - ETA: 0s - loss: 1.3481 - accuracy: 0.5261
1151/1563 [=====================>........] - ETA: 0s - loss: 1.3495 - accuracy: 0.5258
1189/1563 [=====================>........] - ETA: 0s - loss: 1.3505 - accuracy: 0.5255
1227/1563 [======================>.......] - ETA: 0s - loss: 1.3503 - accuracy: 0.5261
1265/1563 [=======================>......] - ETA: 0s - loss: 1.3504 - accuracy: 0.5257
1303/1563 [========================>.....] - ETA: 0s - loss: 1.3503 - accuracy: 0.5260
1342/1563 [========================>.....] - ETA: 0s - loss: 1.3506 - accuracy: 0.5258
1381/1563 [=========================>....] - ETA: 0s - loss: 1.3499 - accuracy: 0.5258
1420/1563 [==========================>...] - ETA: 0s - loss: 1.3493 - accuracy: 0.5261
1458/1563 [==========================>...] - ETA: 0s - loss: 1.3497 - accuracy: 0.5255
1496/1563 [===========================>..] - ETA: 0s - loss: 1.3495 - accuracy: 0.5256
1534/1563 [============================>.] - ETA: 0s - loss: 1.3495 - accuracy: 0.5259
1563/1563 [==============================] - 2s 1ms/step - loss: 1.3492 - accuracy: 0.5261
Epoch 8/10
1/1563 [..............................] - ETA: 2s - loss: 1.1514 - accuracy: 0.5312
39/1563 [..............................] - ETA: 2s - loss: 1.3132 - accuracy: 0.5361
78/1563 [>.............................] - ETA: 1s - loss: 1.3220 - accuracy: 0.5361
117/1563 [=>............................] - ETA: 1s - loss: 1.3234 - accuracy: 0.5304
156/1563 [=>............................] - ETA: 1s - loss: 1.3221 - accuracy: 0.5387
195/1563 [==>...........................] - ETA: 1s - loss: 1.3144 - accuracy: 0.5399
233/1563 [===>..........................] - ETA: 1s - loss: 1.3122 - accuracy: 0.5409
271/1563 [====>.........................] - ETA: 1s - loss: 1.3119 - accuracy: 0.5404
310/1563 [====>.........................] - ETA: 1s - loss: 1.3135 - accuracy: 0.5397
348/1563 [=====>........................] - ETA: 1s - loss: 1.3102 - accuracy: 0.5402
386/1563 [======>.......................] - ETA: 1s - loss: 1.3098 - accuracy: 0.5413
425/1563 [=======>......................] - ETA: 1s - loss: 1.3164 - accuracy: 0.5385
463/1563 [=======>......................] - ETA: 1s - loss: 1.3176 - accuracy: 0.5372
501/1563 [========>.....................] - ETA: 1s - loss: 1.3193 - accuracy: 0.5367
539/1563 [=========>....................] - ETA: 1s - loss: 1.3183 - accuracy: 0.5365
577/1563 [==========>...................] - ETA: 1s - loss: 1.3182 - accuracy: 0.5370
615/1563 [==========>...................] - ETA: 1s - loss: 1.3167 - accuracy: 0.5377
654/1563 [===========>..................] - ETA: 1s - loss: 1.3161 - accuracy: 0.5379
692/1563 [============>.................] - ETA: 1s - loss: 1.3167 - accuracy: 0.5384
731/1563 [=============>................] - ETA: 1s - loss: 1.3193 - accuracy: 0.5370
770/1563 [=============>................] - ETA: 1s - loss: 1.3204 - accuracy: 0.5365
808/1563 [==============>...............] - ETA: 0s - loss: 1.3197 - accuracy: 0.5367
846/1563 [===============>..............] - ETA: 0s - loss: 1.3198 - accuracy: 0.5373
884/1563 [===============>..............] - ETA: 0s - loss: 1.3212 - accuracy: 0.5366
922/1563 [================>.............] - ETA: 0s - loss: 1.3220 - accuracy: 0.5361
960/1563 [=================>............] - ETA: 0s - loss: 1.3224 - accuracy: 0.5361
998/1563 [==================>...........] - ETA: 0s - loss: 1.3203 - accuracy: 0.5370
1037/1563 [==================>...........] - ETA: 0s - loss: 1.3190 - accuracy: 0.5371
1076/1563 [===================>..........] - ETA: 0s - loss: 1.3177 - accuracy: 0.5372
1115/1563 [====================>.........] - ETA: 0s - loss: 1.3182 - accuracy: 0.5363
1153/1563 [=====================>........] - ETA: 0s - loss: 1.3186 - accuracy: 0.5353
1191/1563 [=====================>........] - ETA: 0s - loss: 1.3188 - accuracy: 0.5356
1229/1563 [======================>.......] - ETA: 0s - loss: 1.3175 - accuracy: 0.5359
1267/1563 [=======================>......] - ETA: 0s - loss: 1.3187 - accuracy: 0.5355
1305/1563 [========================>.....] - ETA: 0s - loss: 1.3186 - accuracy: 0.5359
1344/1563 [========================>.....] - ETA: 0s - loss: 1.3182 - accuracy: 0.5358
1382/1563 [=========================>....] - ETA: 0s - loss: 1.3189 - accuracy: 0.5358
1421/1563 [==========================>...] - ETA: 0s - loss: 1.3188 - accuracy: 0.5359
1459/1563 [===========================>..] - ETA: 0s - loss: 1.3158 - accuracy: 0.5373
1497/1563 [===========================>..] - ETA: 0s - loss: 1.3158 - accuracy: 0.5372
1535/1563 [============================>.] - ETA: 0s - loss: 1.3155 - accuracy: 0.5373
1563/1563 [==============================] - 2s 1ms/step - loss: 1.3150 - accuracy: 0.5373
Epoch 9/10
1/1563 [..............................] - ETA: 2s - loss: 1.5591 - accuracy: 0.4375
39/1563 [..............................] - ETA: 2s - loss: 1.2911 - accuracy: 0.5449
77/1563 [>.............................] - ETA: 1s - loss: 1.2704 - accuracy: 0.5475
115/1563 [=>............................] - ETA: 1s - loss: 1.2635 - accuracy: 0.5538
153/1563 [=>............................] - ETA: 1s - loss: 1.2772 - accuracy: 0.5484
191/1563 [==>...........................] - ETA: 1s - loss: 1.2885 - accuracy: 0.5448
229/1563 [===>..........................] - ETA: 1s - loss: 1.2824 - accuracy: 0.5482
267/1563 [====>.........................] - ETA: 1s - loss: 1.2830 - accuracy: 0.5476
305/1563 [====>.........................] - ETA: 1s - loss: 1.2852 - accuracy: 0.5464
343/1563 [=====>........................] - ETA: 1s - loss: 1.2794 - accuracy: 0.5504
381/1563 [======>.......................] - ETA: 1s - loss: 1.2797 - accuracy: 0.5505
420/1563 [=======>......................] - ETA: 1s - loss: 1.2768 - accuracy: 0.5534
459/1563 [=======>......................] - ETA: 1s - loss: 1.2774 - accuracy: 0.5533
498/1563 [========>.....................] - ETA: 1s - loss: 1.2793 - accuracy: 0.5530
536/1563 [=========>....................] - ETA: 1s - loss: 1.2803 - accuracy: 0.5525
574/1563 [==========>...................] - ETA: 1s - loss: 1.2799 - accuracy: 0.5524
612/1563 [==========>...................] - ETA: 1s - loss: 1.2817 - accuracy: 0.5518
650/1563 [===========>..................] - ETA: 1s - loss: 1.2828 - accuracy: 0.5512
689/1563 [============>.................] - ETA: 1s - loss: 1.2836 - accuracy: 0.5507
727/1563 [============>.................] - ETA: 1s - loss: 1.2848 - accuracy: 0.5499
765/1563 [=============>................] - ETA: 1s - loss: 1.2844 - accuracy: 0.5508
804/1563 [==============>...............] - ETA: 1s - loss: 1.2848 - accuracy: 0.5505
842/1563 [===============>..............] - ETA: 0s - loss: 1.2830 - accuracy: 0.5505
880/1563 [===============>..............] - ETA: 0s - loss: 1.2813 - accuracy: 0.5522
918/1563 [================>.............] - ETA: 0s - loss: 1.2792 - accuracy: 0.5529
956/1563 [=================>............] - ETA: 0s - loss: 1.2813 - accuracy: 0.5519
994/1563 [==================>...........] - ETA: 0s - loss: 1.2809 - accuracy: 0.5521
1033/1563 [==================>...........] - ETA: 0s - loss: 1.2805 - accuracy: 0.5519
1072/1563 [===================>..........] - ETA: 0s - loss: 1.2800 - accuracy: 0.5517
1110/1563 [====================>.........] - ETA: 0s - loss: 1.2797 - accuracy: 0.5518
1148/1563 [=====================>........] - ETA: 0s - loss: 1.2799 - accuracy: 0.5519
1186/1563 [=====================>........] - ETA: 0s - loss: 1.2792 - accuracy: 0.5523
1224/1563 [======================>.......] - ETA: 0s - loss: 1.2780 - accuracy: 0.5527
1262/1563 [=======================>......] - ETA: 0s - loss: 1.2785 - accuracy: 0.5522
1300/1563 [=======================>......] - ETA: 0s - loss: 1.2797 - accuracy: 0.5520
1339/1563 [========================>.....] - ETA: 0s - loss: 1.2792 - accuracy: 0.5526
1378/1563 [=========================>....] - ETA: 0s - loss: 1.2806 - accuracy: 0.5521
1416/1563 [==========================>...] - ETA: 0s - loss: 1.2806 - accuracy: 0.5519
1454/1563 [==========================>...] - ETA: 0s - loss: 1.2806 - accuracy: 0.5518
1493/1563 [===========================>..] - ETA: 0s - loss: 1.2795 - accuracy: 0.5520
1531/1563 [============================>.] - ETA: 0s - loss: 1.2805 - accuracy: 0.5515
1563/1563 [==============================] - 2s 1ms/step - loss: 1.2811 - accuracy: 0.5511
Epoch 10/10
1/1563 [..............................] - ETA: 2s - loss: 1.3121 - accuracy: 0.5625
40/1563 [..............................] - ETA: 2s - loss: 1.1949 - accuracy: 0.6078
78/1563 [>.............................] - ETA: 1s - loss: 1.2113 - accuracy: 0.5917
117/1563 [=>............................] - ETA: 1s - loss: 1.2169 - accuracy: 0.5772
156/1563 [=>............................] - ETA: 1s - loss: 1.2235 - accuracy: 0.5723
194/1563 [==>...........................] - ETA: 1s - loss: 1.2240 - accuracy: 0.5714
232/1563 [===>..........................] - ETA: 1s - loss: 1.2188 - accuracy: 0.5746
270/1563 [====>.........................] - ETA: 1s - loss: 1.2340 - accuracy: 0.5696
308/1563 [====>.........................] - ETA: 1s - loss: 1.2363 - accuracy: 0.5684
346/1563 [=====>........................] - ETA: 1s - loss: 1.2339 - accuracy: 0.5675
384/1563 [======>.......................] - ETA: 1s - loss: 1.2385 - accuracy: 0.5653
422/1563 [=======>......................] - ETA: 1s - loss: 1.2381 - accuracy: 0.5654
460/1563 [=======>......................] - ETA: 1s - loss: 1.2340 - accuracy: 0.5668
498/1563 [========>.....................] - ETA: 1s - loss: 1.2346 - accuracy: 0.5662
536/1563 [=========>....................] - ETA: 1s - loss: 1.2354 - accuracy: 0.5655
574/1563 [==========>...................] - ETA: 1s - loss: 1.2376 - accuracy: 0.5641
612/1563 [==========>...................] - ETA: 1s - loss: 1.2390 - accuracy: 0.5632
650/1563 [===========>..................] - ETA: 1s - loss: 1.2404 - accuracy: 0.5630
688/1563 [============>.................] - ETA: 1s - loss: 1.2399 - accuracy: 0.5643
726/1563 [============>.................] - ETA: 1s - loss: 1.2428 - accuracy: 0.5643
764/1563 [=============>................] - ETA: 1s - loss: 1.2431 - accuracy: 0.5637
803/1563 [==============>...............] - ETA: 1s - loss: 1.2442 - accuracy: 0.5629
841/1563 [===============>..............] - ETA: 0s - loss: 1.2417 - accuracy: 0.5644
879/1563 [===============>..............] - ETA: 0s - loss: 1.2449 - accuracy: 0.5637
917/1563 [================>.............] - ETA: 0s - loss: 1.2468 - accuracy: 0.5634
955/1563 [=================>............] - ETA: 0s - loss: 1.2480 - accuracy: 0.5634
993/1563 [==================>...........] - ETA: 0s - loss: 1.2486 - accuracy: 0.5628
1031/1563 [==================>...........] - ETA: 0s - loss: 1.2485 - accuracy: 0.5634
1069/1563 [===================>..........] - ETA: 0s - loss: 1.2493 - accuracy: 0.5627
1107/1563 [====================>.........] - ETA: 0s - loss: 1.2484 - accuracy: 0.5630
1145/1563 [====================>.........] - ETA: 0s - loss: 1.2480 - accuracy: 0.5625
1184/1563 [=====================>........] - ETA: 0s - loss: 1.2475 - accuracy: 0.5627
1222/1563 [======================>.......] - ETA: 0s - loss: 1.2495 - accuracy: 0.5626
1260/1563 [=======================>......] - ETA: 0s - loss: 1.2507 - accuracy: 0.5624
1298/1563 [=======================>......] - ETA: 0s - loss: 1.2509 - accuracy: 0.5623
1336/1563 [========================>.....] - ETA: 0s - loss: 1.2516 - accuracy: 0.5622
1374/1563 [=========================>....] - ETA: 0s - loss: 1.2520 - accuracy: 0.5621
1413/1563 [==========================>...] - ETA: 0s - loss: 1.2509 - accuracy: 0.5622
1452/1563 [==========================>...] - ETA: 0s - loss: 1.2495 - accuracy: 0.5628
1491/1563 [===========================>..] - ETA: 0s - loss: 1.2489 - accuracy: 0.5629
1529/1563 [============================>.] - ETA: 0s - loss: 1.2491 - accuracy: 0.5628
1563/1563 [==============================] - 2s 1ms/step - loss: 1.2495 - accuracy: 0.5623
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
flatten (Flatten) (None, 3072) 0
dense (Dense) (None, 3000) 9219000
dense_1 (Dense) (None, 1000) 3001000
dense_2 (Dense) (None, 10) 10010
=================================================================
Total params: 12,230,010
Trainable params: 12,230,010
Non-trainable params: 0
_________________________________________________________________
None
wall time seconds (without loading test data set)= 39.140345096588135
hydro05
[SSH] completed
[SSH] exit-status: 0
[workspace] $ /bin/sh -xe /tmp/jenkins18045683576075885031.sh
+ scp 'HYDRO_REMOTE:~svchydrojenkins/python/tensorflow_time.txt' /var/lib/jenkins/jobs/tensorflow/workspace
Recording plot data
Saving plot series data from: /var/lib/jenkins/jobs/tensorflow/workspace/tensorflow_time.txt
Finished: SUCCESS
![[Jenkins]](/static/755cd4af/images/svgs/logo.svg){kind=logo}
