Started by timer
Running as SYSTEM
Building in workspace /var/lib/jenkins/jobs/tensorflow/workspace
[SSH] script:
module load anaconda3_gpu
module load cudnn
cd python
rm -f tensorflow_test.txt
srun --nodes=1 --ntasks-per-node=1 --mem=16g \
--time=00:05:00 \
--partition=a100rome \
--account=bbmb-hydro \
--gpus-per-node=1 \
--gpu-bind=closest \
python3 cifar10gpu.py | tee tensorflow_test.txt
YVALUE=`grep "wall time seconds" tensorflow_test.txt | cut -d'=' -f2`
echo YVALUE=$YVALUE > tensorflow_time.txt
[SSH] executing...
srun: job 98520 queued and waiting for resources
srun: job 98520 has been allocated resources
2024-11-24 07:43:02.918527: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-11-24 07:43:03.058755: E tensorflow/stream_executor/cuda/cuda_blas.cc:2981] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-11-24 07:43:04.034566: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /sw/spack/hydrogpu-2022-06/apps/cudnn/8.4.0.27-11.6-gcc-11.3.0-qbhsfav/lib:/sw/spack/hydrogpu-2022-06/apps/cuda/11.7.0-gcc-11.3.0-3ysno6b/lib64:/sw/spack/hydrogpu-2022-06/apps/openmpi/4.1.4-gcc-11.3.0-i6koipj/lib:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib64:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib
2024-11-24 07:43:04.034642: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /sw/spack/hydrogpu-2022-06/apps/cudnn/8.4.0.27-11.6-gcc-11.3.0-qbhsfav/lib:/sw/spack/hydrogpu-2022-06/apps/cuda/11.7.0-gcc-11.3.0-3ysno6b/lib64:/sw/spack/hydrogpu-2022-06/apps/openmpi/4.1.4-gcc-11.3.0-i6koipj/lib:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib64:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib
2024-11-24 07:43:04.034649: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
2024-11-24 07:43:07.104083: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-11-24 07:43:07.478109: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1616] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 79148 MB memory: -> device: 0, name: NVIDIA A100 80GB PCIe, pci bus id: 0000:21:00.0, compute capability: 8.0
(50000, 32, 32, 3) (10000, 32, 32, 3)
Epoch 1/10
2024-11-24 07:43:09.548611: I tensorflow/stream_executor/cuda/cuda_blas.cc:1614] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
1/1563 [..............................] - ETA: 27:59 - loss: 2.3143 - accuracy: 0.0938
37/1563 [..............................] - ETA: 2s - loss: 2.2484 - accuracy: 0.1495
75/1563 [>.............................] - ETA: 2s - loss: 2.1651 - accuracy: 0.1879
113/1563 [=>............................] - ETA: 1s - loss: 2.1094 - accuracy: 0.2185
151/1563 [=>............................] - ETA: 1s - loss: 2.0850 - accuracy: 0.2347
189/1563 [==>...........................] - ETA: 1s - loss: 2.0586 - accuracy: 0.2507
227/1563 [===>..........................] - ETA: 1s - loss: 2.0400 - accuracy: 0.2596
265/1563 [====>.........................] - ETA: 1s - loss: 2.0162 - accuracy: 0.2696
303/1563 [====>.........................] - ETA: 1s - loss: 2.0011 - accuracy: 0.2751
341/1563 [=====>........................] - ETA: 1s - loss: 1.9868 - accuracy: 0.2809
379/1563 [======>.......................] - ETA: 1s - loss: 1.9770 - accuracy: 0.2844
417/1563 [=======>......................] - ETA: 1s - loss: 1.9664 - accuracy: 0.2908
455/1563 [=======>......................] - ETA: 1s - loss: 1.9599 - accuracy: 0.2946
493/1563 [========>.....................] - ETA: 1s - loss: 1.9481 - accuracy: 0.2993
531/1563 [=========>....................] - ETA: 1s - loss: 1.9411 - accuracy: 0.3018
569/1563 [=========>....................] - ETA: 1s - loss: 1.9312 - accuracy: 0.3070
607/1563 [==========>...................] - ETA: 1s - loss: 1.9234 - accuracy: 0.3107
645/1563 [===========>..................] - ETA: 1s - loss: 1.9176 - accuracy: 0.3140
683/1563 [============>.................] - ETA: 1s - loss: 1.9110 - accuracy: 0.3175
721/1563 [============>.................] - ETA: 1s - loss: 1.9064 - accuracy: 0.3192
759/1563 [=============>................] - ETA: 1s - loss: 1.8987 - accuracy: 0.3214
797/1563 [==============>...............] - ETA: 1s - loss: 1.8934 - accuracy: 0.3238
835/1563 [===============>..............] - ETA: 0s - loss: 1.8876 - accuracy: 0.3260
873/1563 [===============>..............] - ETA: 0s - loss: 1.8818 - accuracy: 0.3280
911/1563 [================>.............] - ETA: 0s - loss: 1.8748 - accuracy: 0.3304
949/1563 [=================>............] - ETA: 0s - loss: 1.8701 - accuracy: 0.3322
987/1563 [=================>............] - ETA: 0s - loss: 1.8659 - accuracy: 0.3341
1025/1563 [==================>...........] - ETA: 0s - loss: 1.8620 - accuracy: 0.3358
1063/1563 [===================>..........] - ETA: 0s - loss: 1.8586 - accuracy: 0.3372
1101/1563 [====================>.........] - ETA: 0s - loss: 1.8550 - accuracy: 0.3385
1139/1563 [====================>.........] - ETA: 0s - loss: 1.8503 - accuracy: 0.3400
1177/1563 [=====================>........] - ETA: 0s - loss: 1.8460 - accuracy: 0.3414
1215/1563 [======================>.......] - ETA: 0s - loss: 1.8437 - accuracy: 0.3421
1253/1563 [=======================>......] - ETA: 0s - loss: 1.8388 - accuracy: 0.3441
1291/1563 [=======================>......] - ETA: 0s - loss: 1.8355 - accuracy: 0.3454
1329/1563 [========================>.....] - ETA: 0s - loss: 1.8323 - accuracy: 0.3462
1367/1563 [=========================>....] - ETA: 0s - loss: 1.8277 - accuracy: 0.3478
1405/1563 [=========================>....] - ETA: 0s - loss: 1.8235 - accuracy: 0.3496
1443/1563 [==========================>...] - ETA: 0s - loss: 1.8202 - accuracy: 0.3513
1481/1563 [===========================>..] - ETA: 0s - loss: 1.8169 - accuracy: 0.3531
1519/1563 [============================>.] - ETA: 0s - loss: 1.8139 - accuracy: 0.3541
1557/1563 [============================>.] - ETA: 0s - loss: 1.8102 - accuracy: 0.3552
1563/1563 [==============================] - 3s 1ms/step - loss: 1.8097 - accuracy: 0.3554
Epoch 2/10
1/1563 [..............................] - ETA: 2s - loss: 1.9048 - accuracy: 0.2812
39/1563 [..............................] - ETA: 2s - loss: 1.6667 - accuracy: 0.4030
77/1563 [>.............................] - ETA: 1s - loss: 1.6688 - accuracy: 0.4119
115/1563 [=>............................] - ETA: 1s - loss: 1.6614 - accuracy: 0.4139
153/1563 [=>............................] - ETA: 1s - loss: 1.6516 - accuracy: 0.4158
191/1563 [==>...........................] - ETA: 1s - loss: 1.6530 - accuracy: 0.4190
229/1563 [===>..........................] - ETA: 1s - loss: 1.6469 - accuracy: 0.4225
267/1563 [====>.........................] - ETA: 1s - loss: 1.6419 - accuracy: 0.4247
305/1563 [====>.........................] - ETA: 1s - loss: 1.6464 - accuracy: 0.4247
343/1563 [=====>........................] - ETA: 1s - loss: 1.6429 - accuracy: 0.4244
380/1563 [======>.......................] - ETA: 1s - loss: 1.6409 - accuracy: 0.4252
418/1563 [=======>......................] - ETA: 1s - loss: 1.6397 - accuracy: 0.4262
456/1563 [=======>......................] - ETA: 1s - loss: 1.6378 - accuracy: 0.4252
494/1563 [========>.....................] - ETA: 1s - loss: 1.6370 - accuracy: 0.4247
532/1563 [=========>....................] - ETA: 1s - loss: 1.6348 - accuracy: 0.4253
570/1563 [=========>....................] - ETA: 1s - loss: 1.6346 - accuracy: 0.4248
608/1563 [==========>...................] - ETA: 1s - loss: 1.6332 - accuracy: 0.4251
646/1563 [===========>..................] - ETA: 1s - loss: 1.6336 - accuracy: 0.4257
684/1563 [============>.................] - ETA: 1s - loss: 1.6337 - accuracy: 0.4254
722/1563 [============>.................] - ETA: 1s - loss: 1.6334 - accuracy: 0.4257
760/1563 [=============>................] - ETA: 1s - loss: 1.6325 - accuracy: 0.4260
798/1563 [==============>...............] - ETA: 1s - loss: 1.6315 - accuracy: 0.4266
835/1563 [===============>..............] - ETA: 0s - loss: 1.6320 - accuracy: 0.4267
872/1563 [===============>..............] - ETA: 0s - loss: 1.6325 - accuracy: 0.4269
910/1563 [================>.............] - ETA: 0s - loss: 1.6302 - accuracy: 0.4273
948/1563 [=================>............] - ETA: 0s - loss: 1.6291 - accuracy: 0.4276
986/1563 [=================>............] - ETA: 0s - loss: 1.6293 - accuracy: 0.4277
1024/1563 [==================>...........] - ETA: 0s - loss: 1.6304 - accuracy: 0.4274
1062/1563 [===================>..........] - ETA: 0s - loss: 1.6296 - accuracy: 0.4278
1100/1563 [====================>.........] - ETA: 0s - loss: 1.6304 - accuracy: 0.4272
1138/1563 [====================>.........] - ETA: 0s - loss: 1.6296 - accuracy: 0.4269
1176/1563 [=====================>........] - ETA: 0s - loss: 1.6278 - accuracy: 0.4275
1214/1563 [======================>.......] - ETA: 0s - loss: 1.6265 - accuracy: 0.4281
1252/1563 [=======================>......] - ETA: 0s - loss: 1.6263 - accuracy: 0.4281
1290/1563 [=======================>......] - ETA: 0s - loss: 1.6268 - accuracy: 0.4279
1328/1563 [========================>.....] - ETA: 0s - loss: 1.6257 - accuracy: 0.4277
1366/1563 [=========================>....] - ETA: 0s - loss: 1.6239 - accuracy: 0.4284
1404/1563 [=========================>....] - ETA: 0s - loss: 1.6238 - accuracy: 0.4280
1442/1563 [==========================>...] - ETA: 0s - loss: 1.6225 - accuracy: 0.4286
1480/1563 [===========================>..] - ETA: 0s - loss: 1.6219 - accuracy: 0.4285
1518/1563 [============================>.] - ETA: 0s - loss: 1.6212 - accuracy: 0.4285
1555/1563 [============================>.] - ETA: 0s - loss: 1.6205 - accuracy: 0.4288
1563/1563 [==============================] - 2s 1ms/step - loss: 1.6204 - accuracy: 0.4288
Epoch 3/10
1/1563 [..............................] - ETA: 2s - loss: 1.8260 - accuracy: 0.4062
39/1563 [..............................] - ETA: 2s - loss: 1.5438 - accuracy: 0.4471
77/1563 [>.............................] - ETA: 1s - loss: 1.5558 - accuracy: 0.4476
115/1563 [=>............................] - ETA: 1s - loss: 1.5661 - accuracy: 0.4443
153/1563 [=>............................] - ETA: 1s - loss: 1.5697 - accuracy: 0.4446
191/1563 [==>...........................] - ETA: 1s - loss: 1.5679 - accuracy: 0.4437
229/1563 [===>..........................] - ETA: 1s - loss: 1.5678 - accuracy: 0.4434
267/1563 [====>.........................] - ETA: 1s - loss: 1.5594 - accuracy: 0.4462
305/1563 [====>.........................] - ETA: 1s - loss: 1.5634 - accuracy: 0.4450
343/1563 [=====>........................] - ETA: 1s - loss: 1.5658 - accuracy: 0.4442
381/1563 [======>.......................] - ETA: 1s - loss: 1.5709 - accuracy: 0.4432
419/1563 [=======>......................] - ETA: 1s - loss: 1.5677 - accuracy: 0.4433
457/1563 [=======>......................] - ETA: 1s - loss: 1.5638 - accuracy: 0.4452
495/1563 [========>.....................] - ETA: 1s - loss: 1.5626 - accuracy: 0.4465
533/1563 [=========>....................] - ETA: 1s - loss: 1.5613 - accuracy: 0.4482
571/1563 [=========>....................] - ETA: 1s - loss: 1.5583 - accuracy: 0.4493
609/1563 [==========>...................] - ETA: 1s - loss: 1.5624 - accuracy: 0.4481
647/1563 [===========>..................] - ETA: 1s - loss: 1.5634 - accuracy: 0.4480
685/1563 [============>.................] - ETA: 1s - loss: 1.5597 - accuracy: 0.4494
723/1563 [============>.................] - ETA: 1s - loss: 1.5585 - accuracy: 0.4498
761/1563 [=============>................] - ETA: 1s - loss: 1.5583 - accuracy: 0.4506
799/1563 [==============>...............] - ETA: 1s - loss: 1.5568 - accuracy: 0.4511
837/1563 [===============>..............] - ETA: 0s - loss: 1.5550 - accuracy: 0.4508
875/1563 [===============>..............] - ETA: 0s - loss: 1.5535 - accuracy: 0.4505
913/1563 [================>.............] - ETA: 0s - loss: 1.5532 - accuracy: 0.4503
951/1563 [=================>............] - ETA: 0s - loss: 1.5540 - accuracy: 0.4503
989/1563 [=================>............] - ETA: 0s - loss: 1.5526 - accuracy: 0.4505
1027/1563 [==================>...........] - ETA: 0s - loss: 1.5523 - accuracy: 0.4505
1065/1563 [===================>..........] - ETA: 0s - loss: 1.5506 - accuracy: 0.4510
1103/1563 [====================>.........] - ETA: 0s - loss: 1.5503 - accuracy: 0.4518
1141/1563 [====================>.........] - ETA: 0s - loss: 1.5493 - accuracy: 0.4521
1179/1563 [=====================>........] - ETA: 0s - loss: 1.5484 - accuracy: 0.4527
1217/1563 [======================>.......] - ETA: 0s - loss: 1.5489 - accuracy: 0.4525
1255/1563 [=======================>......] - ETA: 0s - loss: 1.5465 - accuracy: 0.4533
1292/1563 [=======================>......] - ETA: 0s - loss: 1.5463 - accuracy: 0.4536
1330/1563 [========================>.....] - ETA: 0s - loss: 1.5451 - accuracy: 0.4537
1368/1563 [=========================>....] - ETA: 0s - loss: 1.5432 - accuracy: 0.4546
1406/1563 [=========================>....] - ETA: 0s - loss: 1.5439 - accuracy: 0.4544
1444/1563 [==========================>...] - ETA: 0s - loss: 1.5423 - accuracy: 0.4554
1482/1563 [===========================>..] - ETA: 0s - loss: 1.5421 - accuracy: 0.4552
1520/1563 [============================>.] - ETA: 0s - loss: 1.5413 - accuracy: 0.4557
1558/1563 [============================>.] - ETA: 0s - loss: 1.5417 - accuracy: 0.4555
1563/1563 [==============================] - 2s 1ms/step - loss: 1.5414 - accuracy: 0.4556
Epoch 4/10
1/1563 [..............................] - ETA: 2s - loss: 1.6403 - accuracy: 0.4688
39/1563 [..............................] - ETA: 2s - loss: 1.4923 - accuracy: 0.4968
77/1563 [>.............................] - ETA: 1s - loss: 1.4667 - accuracy: 0.5000
115/1563 [=>............................] - ETA: 1s - loss: 1.4899 - accuracy: 0.4894
153/1563 [=>............................] - ETA: 1s - loss: 1.4856 - accuracy: 0.4882
191/1563 [==>...........................] - ETA: 1s - loss: 1.4897 - accuracy: 0.4843
229/1563 [===>..........................] - ETA: 1s - loss: 1.4887 - accuracy: 0.4853
266/1563 [====>.........................] - ETA: 1s - loss: 1.4905 - accuracy: 0.4836
303/1563 [====>.........................] - ETA: 1s - loss: 1.4854 - accuracy: 0.4856
341/1563 [=====>........................] - ETA: 1s - loss: 1.4838 - accuracy: 0.4834
379/1563 [======>.......................] - ETA: 1s - loss: 1.4840 - accuracy: 0.4824
417/1563 [=======>......................] - ETA: 1s - loss: 1.4865 - accuracy: 0.4809
455/1563 [=======>......................] - ETA: 1s - loss: 1.4880 - accuracy: 0.4799
493/1563 [========>.....................] - ETA: 1s - loss: 1.4895 - accuracy: 0.4787
531/1563 [=========>....................] - ETA: 1s - loss: 1.4847 - accuracy: 0.4797
569/1563 [=========>....................] - ETA: 1s - loss: 1.4823 - accuracy: 0.4796
607/1563 [==========>...................] - ETA: 1s - loss: 1.4832 - accuracy: 0.4790
645/1563 [===========>..................] - ETA: 1s - loss: 1.4830 - accuracy: 0.4795
683/1563 [============>.................] - ETA: 1s - loss: 1.4816 - accuracy: 0.4798
721/1563 [============>.................] - ETA: 1s - loss: 1.4824 - accuracy: 0.4792
759/1563 [=============>................] - ETA: 1s - loss: 1.4826 - accuracy: 0.4783
797/1563 [==============>...............] - ETA: 1s - loss: 1.4845 - accuracy: 0.4775
835/1563 [===============>..............] - ETA: 0s - loss: 1.4845 - accuracy: 0.4773
873/1563 [===============>..............] - ETA: 0s - loss: 1.4846 - accuracy: 0.4777
911/1563 [================>.............] - ETA: 0s - loss: 1.4835 - accuracy: 0.4778
949/1563 [=================>............] - ETA: 0s - loss: 1.4829 - accuracy: 0.4781
987/1563 [=================>............] - ETA: 0s - loss: 1.4805 - accuracy: 0.4790
1025/1563 [==================>...........] - ETA: 0s - loss: 1.4803 - accuracy: 0.4789
1063/1563 [===================>..........] - ETA: 0s - loss: 1.4822 - accuracy: 0.4792
1101/1563 [====================>.........] - ETA: 0s - loss: 1.4810 - accuracy: 0.4793
1139/1563 [====================>.........] - ETA: 0s - loss: 1.4796 - accuracy: 0.4793
1177/1563 [=====================>........] - ETA: 0s - loss: 1.4810 - accuracy: 0.4784
1215/1563 [======================>.......] - ETA: 0s - loss: 1.4810 - accuracy: 0.4778
1253/1563 [=======================>......] - ETA: 0s - loss: 1.4824 - accuracy: 0.4773
1291/1563 [=======================>......] - ETA: 0s - loss: 1.4816 - accuracy: 0.4774
1329/1563 [========================>.....] - ETA: 0s - loss: 1.4811 - accuracy: 0.4777
1367/1563 [=========================>....] - ETA: 0s - loss: 1.4816 - accuracy: 0.4773
1405/1563 [=========================>....] - ETA: 0s - loss: 1.4816 - accuracy: 0.4775
1443/1563 [==========================>...] - ETA: 0s - loss: 1.4801 - accuracy: 0.4785
1481/1563 [===========================>..] - ETA: 0s - loss: 1.4796 - accuracy: 0.4786
1519/1563 [============================>.] - ETA: 0s - loss: 1.4802 - accuracy: 0.4789
1557/1563 [============================>.] - ETA: 0s - loss: 1.4794 - accuracy: 0.4790
1563/1563 [==============================] - 2s 1ms/step - loss: 1.4793 - accuracy: 0.4790
Epoch 5/10
1/1563 [..............................] - ETA: 2s - loss: 1.9068 - accuracy: 0.2812
39/1563 [..............................] - ETA: 2s - loss: 1.4710 - accuracy: 0.4720
77/1563 [>.............................] - ETA: 1s - loss: 1.4552 - accuracy: 0.4773
115/1563 [=>............................] - ETA: 1s - loss: 1.4328 - accuracy: 0.4965
153/1563 [=>............................] - ETA: 1s - loss: 1.4305 - accuracy: 0.4945
191/1563 [==>...........................] - ETA: 1s - loss: 1.4317 - accuracy: 0.4964
229/1563 [===>..........................] - ETA: 1s - loss: 1.4330 - accuracy: 0.4939
267/1563 [====>.........................] - ETA: 1s - loss: 1.4330 - accuracy: 0.4926
304/1563 [====>.........................] - ETA: 1s - loss: 1.4357 - accuracy: 0.4914
341/1563 [=====>........................] - ETA: 1s - loss: 1.4331 - accuracy: 0.4940
379/1563 [======>.......................] - ETA: 1s - loss: 1.4322 - accuracy: 0.4958
417/1563 [=======>......................] - ETA: 1s - loss: 1.4316 - accuracy: 0.4951
455/1563 [=======>......................] - ETA: 1s - loss: 1.4286 - accuracy: 0.4977
493/1563 [========>.....................] - ETA: 1s - loss: 1.4283 - accuracy: 0.4974
531/1563 [=========>....................] - ETA: 1s - loss: 1.4300 - accuracy: 0.4955
569/1563 [=========>....................] - ETA: 1s - loss: 1.4282 - accuracy: 0.4953
607/1563 [==========>...................] - ETA: 1s - loss: 1.4326 - accuracy: 0.4943
645/1563 [===========>..................] - ETA: 1s - loss: 1.4318 - accuracy: 0.4945
682/1563 [============>.................] - ETA: 1s - loss: 1.4315 - accuracy: 0.4941
720/1563 [============>.................] - ETA: 1s - loss: 1.4322 - accuracy: 0.4947
758/1563 [=============>................] - ETA: 1s - loss: 1.4337 - accuracy: 0.4937
796/1563 [==============>...............] - ETA: 1s - loss: 1.4321 - accuracy: 0.4947
834/1563 [===============>..............] - ETA: 0s - loss: 1.4324 - accuracy: 0.4946
872/1563 [===============>..............] - ETA: 0s - loss: 1.4329 - accuracy: 0.4943
909/1563 [================>.............] - ETA: 0s - loss: 1.4322 - accuracy: 0.4945
947/1563 [=================>............] - ETA: 0s - loss: 1.4326 - accuracy: 0.4946
985/1563 [=================>............] - ETA: 0s - loss: 1.4334 - accuracy: 0.4946
1023/1563 [==================>...........] - ETA: 0s - loss: 1.4338 - accuracy: 0.4946
1061/1563 [===================>..........] - ETA: 0s - loss: 1.4335 - accuracy: 0.4947
1099/1563 [====================>.........] - ETA: 0s - loss: 1.4330 - accuracy: 0.4950
1137/1563 [====================>.........] - ETA: 0s - loss: 1.4322 - accuracy: 0.4951
1175/1563 [=====================>........] - ETA: 0s - loss: 1.4322 - accuracy: 0.4948
1213/1563 [======================>.......] - ETA: 0s - loss: 1.4326 - accuracy: 0.4950
1251/1563 [=======================>......] - ETA: 0s - loss: 1.4316 - accuracy: 0.4954
1289/1563 [=======================>......] - ETA: 0s - loss: 1.4321 - accuracy: 0.4956
1327/1563 [========================>.....] - ETA: 0s - loss: 1.4316 - accuracy: 0.4955
1365/1563 [=========================>....] - ETA: 0s - loss: 1.4303 - accuracy: 0.4956
1403/1563 [=========================>....] - ETA: 0s - loss: 1.4311 - accuracy: 0.4951
1441/1563 [==========================>...] - ETA: 0s - loss: 1.4313 - accuracy: 0.4951
1479/1563 [===========================>..] - ETA: 0s - loss: 1.4316 - accuracy: 0.4956
1516/1563 [============================>.] - ETA: 0s - loss: 1.4312 - accuracy: 0.4960
1553/1563 [============================>.] - ETA: 0s - loss: 1.4304 - accuracy: 0.4959
1563/1563 [==============================] - 2s 1ms/step - loss: 1.4306 - accuracy: 0.4959
Epoch 6/10
1/1563 [..............................] - ETA: 2s - loss: 1.3034 - accuracy: 0.6250
39/1563 [..............................] - ETA: 2s - loss: 1.3580 - accuracy: 0.5176
77/1563 [>.............................] - ETA: 1s - loss: 1.3508 - accuracy: 0.5292
115/1563 [=>............................] - ETA: 1s - loss: 1.3577 - accuracy: 0.5258
153/1563 [=>............................] - ETA: 1s - loss: 1.3774 - accuracy: 0.5137
191/1563 [==>...........................] - ETA: 1s - loss: 1.3807 - accuracy: 0.5119
229/1563 [===>..........................] - ETA: 1s - loss: 1.3823 - accuracy: 0.5123
267/1563 [====>.........................] - ETA: 1s - loss: 1.3790 - accuracy: 0.5142
305/1563 [====>.........................] - ETA: 1s - loss: 1.3797 - accuracy: 0.5170
343/1563 [=====>........................] - ETA: 1s - loss: 1.3822 - accuracy: 0.5159
381/1563 [======>.......................] - ETA: 1s - loss: 1.3889 - accuracy: 0.5126
419/1563 [=======>......................] - ETA: 1s - loss: 1.3877 - accuracy: 0.5134
457/1563 [=======>......................] - ETA: 1s - loss: 1.3862 - accuracy: 0.5141
495/1563 [========>.....................] - ETA: 1s - loss: 1.3862 - accuracy: 0.5153
533/1563 [=========>....................] - ETA: 1s - loss: 1.3893 - accuracy: 0.5144
571/1563 [=========>....................] - ETA: 1s - loss: 1.3908 - accuracy: 0.5147
609/1563 [==========>...................] - ETA: 1s - loss: 1.3914 - accuracy: 0.5147
647/1563 [===========>..................] - ETA: 1s - loss: 1.3893 - accuracy: 0.5146
685/1563 [============>.................] - ETA: 1s - loss: 1.3880 - accuracy: 0.5152
723/1563 [============>.................] - ETA: 1s - loss: 1.3880 - accuracy: 0.5153
761/1563 [=============>................] - ETA: 1s - loss: 1.3894 - accuracy: 0.5149
799/1563 [==============>...............] - ETA: 1s - loss: 1.3884 - accuracy: 0.5150
837/1563 [===============>..............] - ETA: 0s - loss: 1.3908 - accuracy: 0.5143
875/1563 [===============>..............] - ETA: 0s - loss: 1.3906 - accuracy: 0.5140
913/1563 [================>.............] - ETA: 0s - loss: 1.3908 - accuracy: 0.5140
951/1563 [=================>............] - ETA: 0s - loss: 1.3896 - accuracy: 0.5146
989/1563 [=================>............] - ETA: 0s - loss: 1.3893 - accuracy: 0.5143
1026/1563 [==================>...........] - ETA: 0s - loss: 1.3895 - accuracy: 0.5140
1064/1563 [===================>..........] - ETA: 0s - loss: 1.3883 - accuracy: 0.5136
1102/1563 [====================>.........] - ETA: 0s - loss: 1.3891 - accuracy: 0.5133
1140/1563 [====================>.........] - ETA: 0s - loss: 1.3894 - accuracy: 0.5132
1177/1563 [=====================>........] - ETA: 0s - loss: 1.3884 - accuracy: 0.5140
1214/1563 [======================>.......] - ETA: 0s - loss: 1.3869 - accuracy: 0.5144
1252/1563 [=======================>......] - ETA: 0s - loss: 1.3872 - accuracy: 0.5142
1290/1563 [=======================>......] - ETA: 0s - loss: 1.3867 - accuracy: 0.5141
1328/1563 [========================>.....] - ETA: 0s - loss: 1.3861 - accuracy: 0.5142
1366/1563 [=========================>....] - ETA: 0s - loss: 1.3865 - accuracy: 0.5137
1404/1563 [=========================>....] - ETA: 0s - loss: 1.3858 - accuracy: 0.5139
1442/1563 [==========================>...] - ETA: 0s - loss: 1.3855 - accuracy: 0.5140
1480/1563 [===========================>..] - ETA: 0s - loss: 1.3870 - accuracy: 0.5136
1518/1563 [============================>.] - ETA: 0s - loss: 1.3876 - accuracy: 0.5132
1556/1563 [============================>.] - ETA: 0s - loss: 1.3867 - accuracy: 0.5136
1563/1563 [==============================] - 2s 1ms/step - loss: 1.3874 - accuracy: 0.5131
Epoch 7/10
1/1563 [..............................] - ETA: 2s - loss: 1.3519 - accuracy: 0.5312
39/1563 [..............................] - ETA: 2s - loss: 1.3452 - accuracy: 0.5280
77/1563 [>.............................] - ETA: 1s - loss: 1.3247 - accuracy: 0.5341
115/1563 [=>............................] - ETA: 1s - loss: 1.3397 - accuracy: 0.5304
153/1563 [=>............................] - ETA: 1s - loss: 1.3388 - accuracy: 0.5304
191/1563 [==>...........................] - ETA: 1s - loss: 1.3446 - accuracy: 0.5286
229/1563 [===>..........................] - ETA: 1s - loss: 1.3407 - accuracy: 0.5273
266/1563 [====>.........................] - ETA: 1s - loss: 1.3428 - accuracy: 0.5261
304/1563 [====>.........................] - ETA: 1s - loss: 1.3372 - accuracy: 0.5306
342/1563 [=====>........................] - ETA: 1s - loss: 1.3411 - accuracy: 0.5292
380/1563 [======>.......................] - ETA: 1s - loss: 1.3411 - accuracy: 0.5275
418/1563 [=======>......................] - ETA: 1s - loss: 1.3451 - accuracy: 0.5244
456/1563 [=======>......................] - ETA: 1s - loss: 1.3427 - accuracy: 0.5259
494/1563 [========>.....................] - ETA: 1s - loss: 1.3441 - accuracy: 0.5276
532/1563 [=========>....................] - ETA: 1s - loss: 1.3429 - accuracy: 0.5294
569/1563 [=========>....................] - ETA: 1s - loss: 1.3412 - accuracy: 0.5311
607/1563 [==========>...................] - ETA: 1s - loss: 1.3445 - accuracy: 0.5301
645/1563 [===========>..................] - ETA: 1s - loss: 1.3432 - accuracy: 0.5313
683/1563 [============>.................] - ETA: 1s - loss: 1.3446 - accuracy: 0.5300
721/1563 [============>.................] - ETA: 1s - loss: 1.3452 - accuracy: 0.5303
759/1563 [=============>................] - ETA: 1s - loss: 1.3448 - accuracy: 0.5302
797/1563 [==============>...............] - ETA: 1s - loss: 1.3433 - accuracy: 0.5300
835/1563 [===============>..............] - ETA: 0s - loss: 1.3456 - accuracy: 0.5285
873/1563 [===============>..............] - ETA: 0s - loss: 1.3465 - accuracy: 0.5275
911/1563 [================>.............] - ETA: 0s - loss: 1.3472 - accuracy: 0.5270
949/1563 [=================>............] - ETA: 0s - loss: 1.3488 - accuracy: 0.5265
987/1563 [=================>............] - ETA: 0s - loss: 1.3488 - accuracy: 0.5271
1025/1563 [==================>...........] - ETA: 0s - loss: 1.3506 - accuracy: 0.5265
1063/1563 [===================>..........] - ETA: 0s - loss: 1.3514 - accuracy: 0.5262
1101/1563 [====================>.........] - ETA: 0s - loss: 1.3511 - accuracy: 0.5262
1139/1563 [====================>.........] - ETA: 0s - loss: 1.3517 - accuracy: 0.5263
1176/1563 [=====================>........] - ETA: 0s - loss: 1.3511 - accuracy: 0.5265
1214/1563 [======================>.......] - ETA: 0s - loss: 1.3498 - accuracy: 0.5275
1251/1563 [=======================>......] - ETA: 0s - loss: 1.3519 - accuracy: 0.5266
1289/1563 [=======================>......] - ETA: 0s - loss: 1.3499 - accuracy: 0.5272
1327/1563 [========================>.....] - ETA: 0s - loss: 1.3499 - accuracy: 0.5267
1365/1563 [=========================>....] - ETA: 0s - loss: 1.3504 - accuracy: 0.5268
1403/1563 [=========================>....] - ETA: 0s - loss: 1.3491 - accuracy: 0.5272
1441/1563 [==========================>...] - ETA: 0s - loss: 1.3490 - accuracy: 0.5275
1479/1563 [===========================>..] - ETA: 0s - loss: 1.3493 - accuracy: 0.5270
1517/1563 [============================>.] - ETA: 0s - loss: 1.3486 - accuracy: 0.5275
1554/1563 [============================>.] - ETA: 0s - loss: 1.3485 - accuracy: 0.5273
1563/1563 [==============================] - 2s 1ms/step - loss: 1.3483 - accuracy: 0.5274
Epoch 8/10
1/1563 [..............................] - ETA: 2s - loss: 1.2402 - accuracy: 0.5000
39/1563 [..............................] - ETA: 2s - loss: 1.3210 - accuracy: 0.5312
77/1563 [>.............................] - ETA: 1s - loss: 1.2964 - accuracy: 0.5450
115/1563 [=>............................] - ETA: 1s - loss: 1.3006 - accuracy: 0.5386
154/1563 [=>............................] - ETA: 1s - loss: 1.2983 - accuracy: 0.5384
192/1563 [==>...........................] - ETA: 1s - loss: 1.2912 - accuracy: 0.5420
230/1563 [===>..........................] - ETA: 1s - loss: 1.2989 - accuracy: 0.5410
268/1563 [====>.........................] - ETA: 1s - loss: 1.2937 - accuracy: 0.5444
306/1563 [====>.........................] - ETA: 1s - loss: 1.2937 - accuracy: 0.5445
343/1563 [=====>........................] - ETA: 1s - loss: 1.2995 - accuracy: 0.5414
381/1563 [======>.......................] - ETA: 1s - loss: 1.3028 - accuracy: 0.5400
419/1563 [=======>......................] - ETA: 1s - loss: 1.3046 - accuracy: 0.5399
457/1563 [=======>......................] - ETA: 1s - loss: 1.3066 - accuracy: 0.5403
494/1563 [========>.....................] - ETA: 1s - loss: 1.3082 - accuracy: 0.5399
532/1563 [=========>....................] - ETA: 1s - loss: 1.3096 - accuracy: 0.5402
570/1563 [=========>....................] - ETA: 1s - loss: 1.3074 - accuracy: 0.5418
608/1563 [==========>...................] - ETA: 1s - loss: 1.3075 - accuracy: 0.5422
646/1563 [===========>..................] - ETA: 1s - loss: 1.3103 - accuracy: 0.5411
684/1563 [============>.................] - ETA: 1s - loss: 1.3121 - accuracy: 0.5407
722/1563 [============>.................] - ETA: 1s - loss: 1.3105 - accuracy: 0.5418
760/1563 [=============>................] - ETA: 1s - loss: 1.3085 - accuracy: 0.5429
798/1563 [==============>...............] - ETA: 1s - loss: 1.3102 - accuracy: 0.5415
836/1563 [===============>..............] - ETA: 0s - loss: 1.3112 - accuracy: 0.5403
874/1563 [===============>..............] - ETA: 0s - loss: 1.3129 - accuracy: 0.5394
912/1563 [================>.............] - ETA: 0s - loss: 1.3121 - accuracy: 0.5400
950/1563 [=================>............] - ETA: 0s - loss: 1.3132 - accuracy: 0.5395
988/1563 [=================>............] - ETA: 0s - loss: 1.3139 - accuracy: 0.5396
1026/1563 [==================>...........] - ETA: 0s - loss: 1.3136 - accuracy: 0.5398
1064/1563 [===================>..........] - ETA: 0s - loss: 1.3127 - accuracy: 0.5401
1102/1563 [====================>.........] - ETA: 0s - loss: 1.3133 - accuracy: 0.5394
1140/1563 [====================>.........] - ETA: 0s - loss: 1.3153 - accuracy: 0.5383
1178/1563 [=====================>........] - ETA: 0s - loss: 1.3131 - accuracy: 0.5392
1216/1563 [======================>.......] - ETA: 0s - loss: 1.3137 - accuracy: 0.5387
1254/1563 [=======================>......] - ETA: 0s - loss: 1.3138 - accuracy: 0.5384
1293/1563 [=======================>......] - ETA: 0s - loss: 1.3139 - accuracy: 0.5384
1331/1563 [========================>.....] - ETA: 0s - loss: 1.3152 - accuracy: 0.5381
1369/1563 [=========================>....] - ETA: 0s - loss: 1.3147 - accuracy: 0.5384
1408/1563 [==========================>...] - ETA: 0s - loss: 1.3145 - accuracy: 0.5382
1446/1563 [==========================>...] - ETA: 0s - loss: 1.3148 - accuracy: 0.5383
1484/1563 [===========================>..] - ETA: 0s - loss: 1.3136 - accuracy: 0.5387
1522/1563 [============================>.] - ETA: 0s - loss: 1.3147 - accuracy: 0.5382
1560/1563 [============================>.] - ETA: 0s - loss: 1.3149 - accuracy: 0.5381
1563/1563 [==============================] - 2s 1ms/step - loss: 1.3148 - accuracy: 0.5382
Epoch 9/10
1/1563 [..............................] - ETA: 2s - loss: 1.5425 - accuracy: 0.3438
39/1563 [..............................] - ETA: 2s - loss: 1.3132 - accuracy: 0.5401
77/1563 [>.............................] - ETA: 1s - loss: 1.2935 - accuracy: 0.5491
115/1563 [=>............................] - ETA: 1s - loss: 1.2960 - accuracy: 0.5437
153/1563 [=>............................] - ETA: 1s - loss: 1.2994 - accuracy: 0.5433
191/1563 [==>...........................] - ETA: 1s - loss: 1.2953 - accuracy: 0.5452
229/1563 [===>..........................] - ETA: 1s - loss: 1.2946 - accuracy: 0.5437
267/1563 [====>.........................] - ETA: 1s - loss: 1.2879 - accuracy: 0.5459
305/1563 [====>.........................] - ETA: 1s - loss: 1.2860 - accuracy: 0.5444
343/1563 [=====>........................] - ETA: 1s - loss: 1.2925 - accuracy: 0.5419
380/1563 [======>.......................] - ETA: 1s - loss: 1.2912 - accuracy: 0.5431
417/1563 [=======>......................] - ETA: 1s - loss: 1.2886 - accuracy: 0.5447
454/1563 [=======>......................] - ETA: 1s - loss: 1.2906 - accuracy: 0.5441
491/1563 [========>.....................] - ETA: 1s - loss: 1.2911 - accuracy: 0.5435
528/1563 [=========>....................] - ETA: 1s - loss: 1.2889 - accuracy: 0.5451
565/1563 [=========>....................] - ETA: 1s - loss: 1.2882 - accuracy: 0.5459
602/1563 [==========>...................] - ETA: 1s - loss: 1.2892 - accuracy: 0.5453
640/1563 [===========>..................] - ETA: 1s - loss: 1.2898 - accuracy: 0.5456
678/1563 [============>.................] - ETA: 1s - loss: 1.2893 - accuracy: 0.5454
715/1563 [============>.................] - ETA: 1s - loss: 1.2882 - accuracy: 0.5454
752/1563 [=============>................] - ETA: 1s - loss: 1.2867 - accuracy: 0.5464
789/1563 [==============>...............] - ETA: 1s - loss: 1.2883 - accuracy: 0.5456
826/1563 [==============>...............] - ETA: 0s - loss: 1.2885 - accuracy: 0.5452
863/1563 [===============>..............] - ETA: 0s - loss: 1.2882 - accuracy: 0.5463
900/1563 [================>.............] - ETA: 0s - loss: 1.2883 - accuracy: 0.5461
937/1563 [================>.............] - ETA: 0s - loss: 1.2885 - accuracy: 0.5457
974/1563 [=================>............] - ETA: 0s - loss: 1.2865 - accuracy: 0.5467
1012/1563 [==================>...........] - ETA: 0s - loss: 1.2848 - accuracy: 0.5472
1049/1563 [===================>..........] - ETA: 0s - loss: 1.2849 - accuracy: 0.5473
1087/1563 [===================>..........] - ETA: 0s - loss: 1.2855 - accuracy: 0.5471
1124/1563 [====================>.........] - ETA: 0s - loss: 1.2858 - accuracy: 0.5471
1161/1563 [=====================>........] - ETA: 0s - loss: 1.2862 - accuracy: 0.5468
1199/1563 [======================>.......] - ETA: 0s - loss: 1.2850 - accuracy: 0.5477
1237/1563 [======================>.......] - ETA: 0s - loss: 1.2851 - accuracy: 0.5474
1274/1563 [=======================>......] - ETA: 0s - loss: 1.2857 - accuracy: 0.5475
1311/1563 [========================>.....] - ETA: 0s - loss: 1.2859 - accuracy: 0.5474
1348/1563 [========================>.....] - ETA: 0s - loss: 1.2845 - accuracy: 0.5481
1385/1563 [=========================>....] - ETA: 0s - loss: 1.2824 - accuracy: 0.5489
1422/1563 [==========================>...] - ETA: 0s - loss: 1.2840 - accuracy: 0.5480
1459/1563 [===========================>..] - ETA: 0s - loss: 1.2847 - accuracy: 0.5479
1497/1563 [===========================>..] - ETA: 0s - loss: 1.2829 - accuracy: 0.5485
1534/1563 [============================>.] - ETA: 0s - loss: 1.2821 - accuracy: 0.5486
1563/1563 [==============================] - 2s 1ms/step - loss: 1.2818 - accuracy: 0.5490
Epoch 10/10
1/1563 [..............................] - ETA: 2s - loss: 1.2327 - accuracy: 0.5000
38/1563 [..............................] - ETA: 2s - loss: 1.2427 - accuracy: 0.5518
76/1563 [>.............................] - ETA: 2s - loss: 1.2278 - accuracy: 0.5576
113/1563 [=>............................] - ETA: 1s - loss: 1.2328 - accuracy: 0.5647
150/1563 [=>............................] - ETA: 1s - loss: 1.2284 - accuracy: 0.5685
187/1563 [==>...........................] - ETA: 1s - loss: 1.2282 - accuracy: 0.5653
224/1563 [===>..........................] - ETA: 1s - loss: 1.2344 - accuracy: 0.5640
261/1563 [====>.........................] - ETA: 1s - loss: 1.2358 - accuracy: 0.5629
298/1563 [====>.........................] - ETA: 1s - loss: 1.2382 - accuracy: 0.5630
336/1563 [=====>........................] - ETA: 1s - loss: 1.2375 - accuracy: 0.5636
373/1563 [======>.......................] - ETA: 1s - loss: 1.2393 - accuracy: 0.5631
410/1563 [======>.......................] - ETA: 1s - loss: 1.2408 - accuracy: 0.5626
447/1563 [=======>......................] - ETA: 1s - loss: 1.2432 - accuracy: 0.5647
484/1563 [========>.....................] - ETA: 1s - loss: 1.2444 - accuracy: 0.5628
521/1563 [=========>....................] - ETA: 1s - loss: 1.2437 - accuracy: 0.5629
558/1563 [=========>....................] - ETA: 1s - loss: 1.2424 - accuracy: 0.5640
595/1563 [==========>...................] - ETA: 1s - loss: 1.2423 - accuracy: 0.5637
632/1563 [===========>..................] - ETA: 1s - loss: 1.2453 - accuracy: 0.5632
669/1563 [===========>..................] - ETA: 1s - loss: 1.2460 - accuracy: 0.5635
706/1563 [============>.................] - ETA: 1s - loss: 1.2453 - accuracy: 0.5632
743/1563 [=============>................] - ETA: 1s - loss: 1.2452 - accuracy: 0.5634
781/1563 [=============>................] - ETA: 1s - loss: 1.2439 - accuracy: 0.5636
819/1563 [==============>...............] - ETA: 1s - loss: 1.2422 - accuracy: 0.5636
856/1563 [===============>..............] - ETA: 0s - loss: 1.2436 - accuracy: 0.5639
894/1563 [================>.............] - ETA: 0s - loss: 1.2472 - accuracy: 0.5628
932/1563 [================>.............] - ETA: 0s - loss: 1.2467 - accuracy: 0.5629
970/1563 [=================>............] - ETA: 0s - loss: 1.2476 - accuracy: 0.5626
1007/1563 [==================>...........] - ETA: 0s - loss: 1.2489 - accuracy: 0.5623
1044/1563 [===================>..........] - ETA: 0s - loss: 1.2500 - accuracy: 0.5623
1082/1563 [===================>..........] - ETA: 0s - loss: 1.2508 - accuracy: 0.5619
1120/1563 [====================>.........] - ETA: 0s - loss: 1.2498 - accuracy: 0.5621
1157/1563 [=====================>........] - ETA: 0s - loss: 1.2503 - accuracy: 0.5616
1194/1563 [=====================>........] - ETA: 0s - loss: 1.2504 - accuracy: 0.5614
1231/1563 [======================>.......] - ETA: 0s - loss: 1.2510 - accuracy: 0.5610
1268/1563 [=======================>......] - ETA: 0s - loss: 1.2516 - accuracy: 0.5605
1305/1563 [========================>.....] - ETA: 0s - loss: 1.2516 - accuracy: 0.5603
1342/1563 [========================>.....] - ETA: 0s - loss: 1.2524 - accuracy: 0.5600
1379/1563 [=========================>....] - ETA: 0s - loss: 1.2527 - accuracy: 0.5598
1417/1563 [==========================>...] - ETA: 0s - loss: 1.2525 - accuracy: 0.5601
1455/1563 [==========================>...] - ETA: 0s - loss: 1.2535 - accuracy: 0.5599
1492/1563 [===========================>..] - ETA: 0s - loss: 1.2540 - accuracy: 0.5597
1529/1563 [============================>.] - ETA: 0s - loss: 1.2532 - accuracy: 0.5600
1563/1563 [==============================] - 2s 1ms/step - loss: 1.2532 - accuracy: 0.5602
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
flatten (Flatten) (None, 3072) 0
dense (Dense) (None, 3000) 9219000
dense_1 (Dense) (None, 1000) 3001000
dense_2 (Dense) (None, 10) 10010
=================================================================
Total params: 12,230,010
Trainable params: 12,230,010
Non-trainable params: 0
_________________________________________________________________
None
wall time seconds (without loading test data set)= 23.855648279190063
hydro04
[SSH] completed
[SSH] exit-status: 0
[workspace] $ /bin/sh -xe /tmp/jenkins7810029610055880196.sh
+ scp 'HYDRO_REMOTE:~svchydrojenkins/python/tensorflow_time.txt' /var/lib/jenkins/jobs/tensorflow/workspace
Recording plot data
Saving plot series data from: /var/lib/jenkins/jobs/tensorflow/workspace/tensorflow_time.txt
Finished: SUCCESS
![[Jenkins]](/static/755cd4af/images/svgs/logo.svg){kind=logo}
