Started by timer
Running as SYSTEM
Building in workspace /var/lib/jenkins/jobs/tensorflow/workspace
[SSH] script:
module load anaconda3_gpu
module load cudnn
cd python
rm -f tensorflow_test.txt
srun --nodes=1 --ntasks-per-node=1 --mem=16g \
--time=00:05:00 \
--partition=a100rome \
--account=bbmb-hydro \
--gpus-per-node=1 \
--gpu-bind=closest \
python3 cifar10gpu.py | tee tensorflow_test.txt
YVALUE=`grep "wall time seconds" tensorflow_test.txt | cut -d'=' -f2`
echo YVALUE=$YVALUE > tensorflow_time.txt
[SSH] executing...
srun: job 85470 queued and waiting for resources
srun: job 85470 has been allocated resources
2024-10-19 07:43:07.824801: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-10-19 07:43:31.891041: E tensorflow/stream_executor/cuda/cuda_blas.cc:2981] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-10-19 07:44:05.911195: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /sw/spack/hydrogpu-2022-06/apps/cudnn/8.4.0.27-11.6-gcc-11.3.0-qbhsfav/lib:/sw/spack/hydrogpu-2022-06/apps/cuda/11.7.0-gcc-11.3.0-3ysno6b/lib64:/sw/spack/hydrogpu-2022-06/apps/openmpi/4.1.4-gcc-11.3.0-i6koipj/lib:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib64:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib
2024-10-19 07:44:05.911963: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /sw/spack/hydrogpu-2022-06/apps/cudnn/8.4.0.27-11.6-gcc-11.3.0-qbhsfav/lib:/sw/spack/hydrogpu-2022-06/apps/cuda/11.7.0-gcc-11.3.0-3ysno6b/lib64:/sw/spack/hydrogpu-2022-06/apps/openmpi/4.1.4-gcc-11.3.0-i6koipj/lib:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib64:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib
2024-10-19 07:44:05.911971: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
2024-10-19 07:44:58.674092: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-10-19 07:45:01.246047: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1616] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 79148 MB memory: -> device: 0, name: NVIDIA A100 80GB PCIe, pci bus id: 0000:21:00.0, compute capability: 8.0
(50000, 32, 32, 3) (10000, 32, 32, 3)
Epoch 1/10
2024-10-19 07:45:08.411700: I tensorflow/stream_executor/cuda/cuda_blas.cc:1614] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
1/1563 [..............................] - ETA: 1:39:02 - loss: 2.4576 - accuracy: 0.0312
37/1563 [..............................] - ETA: 2s - loss: 2.2385 - accuracy: 0.1698
74/1563 [>.............................] - ETA: 2s - loss: 2.1611 - accuracy: 0.1964
112/1563 [=>............................] - ETA: 1s - loss: 2.1182 - accuracy: 0.2174
150/1563 [=>............................] - ETA: 1s - loss: 2.0844 - accuracy: 0.2294
188/1563 [==>...........................] - ETA: 1s - loss: 2.0565 - accuracy: 0.2440
225/1563 [===>..........................] - ETA: 1s - loss: 2.0319 - accuracy: 0.2542
263/1563 [====>.........................] - ETA: 1s - loss: 2.0150 - accuracy: 0.2622
301/1563 [====>.........................] - ETA: 1s - loss: 2.0002 - accuracy: 0.2676
339/1563 [=====>........................] - ETA: 1s - loss: 1.9899 - accuracy: 0.2737
377/1563 [======>.......................] - ETA: 1s - loss: 1.9789 - accuracy: 0.2805
415/1563 [======>.......................] - ETA: 1s - loss: 1.9681 - accuracy: 0.2849
453/1563 [=======>......................] - ETA: 1s - loss: 1.9570 - accuracy: 0.2890
491/1563 [========>.....................] - ETA: 1s - loss: 1.9476 - accuracy: 0.2937
529/1563 [=========>....................] - ETA: 1s - loss: 1.9390 - accuracy: 0.2981
567/1563 [=========>....................] - ETA: 1s - loss: 1.9324 - accuracy: 0.3010
605/1563 [==========>...................] - ETA: 1s - loss: 1.9233 - accuracy: 0.3048
643/1563 [===========>..................] - ETA: 1s - loss: 1.9146 - accuracy: 0.3092
681/1563 [============>.................] - ETA: 1s - loss: 1.9074 - accuracy: 0.3124
719/1563 [============>.................] - ETA: 1s - loss: 1.9019 - accuracy: 0.3149
757/1563 [=============>................] - ETA: 1s - loss: 1.8944 - accuracy: 0.3177
795/1563 [==============>...............] - ETA: 1s - loss: 1.8874 - accuracy: 0.3202
833/1563 [==============>...............] - ETA: 0s - loss: 1.8832 - accuracy: 0.3216
871/1563 [===============>..............] - ETA: 0s - loss: 1.8787 - accuracy: 0.3234
909/1563 [================>.............] - ETA: 0s - loss: 1.8744 - accuracy: 0.3254
947/1563 [=================>............] - ETA: 0s - loss: 1.8699 - accuracy: 0.3271
985/1563 [=================>............] - ETA: 0s - loss: 1.8648 - accuracy: 0.3304
1023/1563 [==================>...........] - ETA: 0s - loss: 1.8599 - accuracy: 0.3317
1061/1563 [===================>..........] - ETA: 0s - loss: 1.8565 - accuracy: 0.3335
1099/1563 [====================>.........] - ETA: 0s - loss: 1.8517 - accuracy: 0.3358
1137/1563 [====================>.........] - ETA: 0s - loss: 1.8467 - accuracy: 0.3380
1175/1563 [=====================>........] - ETA: 0s - loss: 1.8437 - accuracy: 0.3391
1213/1563 [======================>.......] - ETA: 0s - loss: 1.8402 - accuracy: 0.3406
1251/1563 [=======================>......] - ETA: 0s - loss: 1.8357 - accuracy: 0.3426
1289/1563 [=======================>......] - ETA: 0s - loss: 1.8332 - accuracy: 0.3439
1327/1563 [========================>.....] - ETA: 0s - loss: 1.8293 - accuracy: 0.3460
1365/1563 [=========================>....] - ETA: 0s - loss: 1.8259 - accuracy: 0.3474
1403/1563 [=========================>....] - ETA: 0s - loss: 1.8226 - accuracy: 0.3486
1441/1563 [==========================>...] - ETA: 0s - loss: 1.8187 - accuracy: 0.3503
1479/1563 [===========================>..] - ETA: 0s - loss: 1.8162 - accuracy: 0.3513
1517/1563 [============================>.] - ETA: 0s - loss: 1.8130 - accuracy: 0.3529
1555/1563 [============================>.] - ETA: 0s - loss: 1.8106 - accuracy: 0.3541
1563/1563 [==============================] - 6s 1ms/step - loss: 1.8105 - accuracy: 0.3543
Epoch 2/10
1/1563 [..............................] - ETA: 2s - loss: 1.6422 - accuracy: 0.4688
39/1563 [..............................] - ETA: 2s - loss: 1.6774 - accuracy: 0.4191
77/1563 [>.............................] - ETA: 1s - loss: 1.6697 - accuracy: 0.4156
115/1563 [=>............................] - ETA: 1s - loss: 1.6699 - accuracy: 0.4120
153/1563 [=>............................] - ETA: 1s - loss: 1.6737 - accuracy: 0.4081
191/1563 [==>...........................] - ETA: 1s - loss: 1.6634 - accuracy: 0.4116
229/1563 [===>..........................] - ETA: 1s - loss: 1.6608 - accuracy: 0.4162
267/1563 [====>.........................] - ETA: 1s - loss: 1.6597 - accuracy: 0.4163
305/1563 [====>.........................] - ETA: 1s - loss: 1.6581 - accuracy: 0.4148
343/1563 [=====>........................] - ETA: 1s - loss: 1.6588 - accuracy: 0.4131
381/1563 [======>.......................] - ETA: 1s - loss: 1.6557 - accuracy: 0.4122
419/1563 [=======>......................] - ETA: 1s - loss: 1.6567 - accuracy: 0.4122
457/1563 [=======>......................] - ETA: 1s - loss: 1.6535 - accuracy: 0.4129
495/1563 [========>.....................] - ETA: 1s - loss: 1.6536 - accuracy: 0.4111
533/1563 [=========>....................] - ETA: 1s - loss: 1.6510 - accuracy: 0.4123
571/1563 [=========>....................] - ETA: 1s - loss: 1.6528 - accuracy: 0.4122
609/1563 [==========>...................] - ETA: 1s - loss: 1.6517 - accuracy: 0.4131
647/1563 [===========>..................] - ETA: 1s - loss: 1.6465 - accuracy: 0.4147
685/1563 [============>.................] - ETA: 1s - loss: 1.6460 - accuracy: 0.4149
723/1563 [============>.................] - ETA: 1s - loss: 1.6449 - accuracy: 0.4156
761/1563 [=============>................] - ETA: 1s - loss: 1.6440 - accuracy: 0.4160
799/1563 [==============>...............] - ETA: 1s - loss: 1.6435 - accuracy: 0.4163
837/1563 [===============>..............] - ETA: 0s - loss: 1.6413 - accuracy: 0.4179
875/1563 [===============>..............] - ETA: 0s - loss: 1.6406 - accuracy: 0.4182
913/1563 [================>.............] - ETA: 0s - loss: 1.6400 - accuracy: 0.4183
951/1563 [=================>............] - ETA: 0s - loss: 1.6394 - accuracy: 0.4191
989/1563 [=================>............] - ETA: 0s - loss: 1.6399 - accuracy: 0.4186
1026/1563 [==================>...........] - ETA: 0s - loss: 1.6376 - accuracy: 0.4204
1064/1563 [===================>..........] - ETA: 0s - loss: 1.6362 - accuracy: 0.4212
1101/1563 [====================>.........] - ETA: 0s - loss: 1.6348 - accuracy: 0.4213
1138/1563 [====================>.........] - ETA: 0s - loss: 1.6327 - accuracy: 0.4224
1175/1563 [=====================>........] - ETA: 0s - loss: 1.6321 - accuracy: 0.4226
1213/1563 [======================>.......] - ETA: 0s - loss: 1.6322 - accuracy: 0.4227
1251/1563 [=======================>......] - ETA: 0s - loss: 1.6308 - accuracy: 0.4231
1289/1563 [=======================>......] - ETA: 0s - loss: 1.6286 - accuracy: 0.4233
1327/1563 [========================>.....] - ETA: 0s - loss: 1.6282 - accuracy: 0.4233
1365/1563 [=========================>....] - ETA: 0s - loss: 1.6287 - accuracy: 0.4230
1402/1563 [=========================>....] - ETA: 0s - loss: 1.6264 - accuracy: 0.4237
1439/1563 [==========================>...] - ETA: 0s - loss: 1.6258 - accuracy: 0.4237
1476/1563 [===========================>..] - ETA: 0s - loss: 1.6246 - accuracy: 0.4241
1513/1563 [============================>.] - ETA: 0s - loss: 1.6239 - accuracy: 0.4248
1550/1563 [============================>.] - ETA: 0s - loss: 1.6226 - accuracy: 0.4253
1563/1563 [==============================] - 2s 1ms/step - loss: 1.6222 - accuracy: 0.4254
Epoch 3/10
1/1563 [..............................] - ETA: 2s - loss: 1.6705 - accuracy: 0.3125
39/1563 [..............................] - ETA: 2s - loss: 1.5360 - accuracy: 0.4583
77/1563 [>.............................] - ETA: 1s - loss: 1.5805 - accuracy: 0.4420
115/1563 [=>............................] - ETA: 1s - loss: 1.5743 - accuracy: 0.4437
153/1563 [=>............................] - ETA: 1s - loss: 1.5769 - accuracy: 0.4420
191/1563 [==>...........................] - ETA: 1s - loss: 1.5748 - accuracy: 0.4434
229/1563 [===>..........................] - ETA: 1s - loss: 1.5668 - accuracy: 0.4446
267/1563 [====>.........................] - ETA: 1s - loss: 1.5668 - accuracy: 0.4449
305/1563 [====>.........................] - ETA: 1s - loss: 1.5710 - accuracy: 0.4459
342/1563 [=====>........................] - ETA: 1s - loss: 1.5699 - accuracy: 0.4474
380/1563 [======>.......................] - ETA: 1s - loss: 1.5658 - accuracy: 0.4489
418/1563 [=======>......................] - ETA: 1s - loss: 1.5610 - accuracy: 0.4507
456/1563 [=======>......................] - ETA: 1s - loss: 1.5620 - accuracy: 0.4504
494/1563 [========>.....................] - ETA: 1s - loss: 1.5604 - accuracy: 0.4497
532/1563 [=========>....................] - ETA: 1s - loss: 1.5589 - accuracy: 0.4517
570/1563 [=========>....................] - ETA: 1s - loss: 1.5579 - accuracy: 0.4518
608/1563 [==========>...................] - ETA: 1s - loss: 1.5557 - accuracy: 0.4521
646/1563 [===========>..................] - ETA: 1s - loss: 1.5564 - accuracy: 0.4517
684/1563 [============>.................] - ETA: 1s - loss: 1.5560 - accuracy: 0.4518
721/1563 [============>.................] - ETA: 1s - loss: 1.5541 - accuracy: 0.4518
759/1563 [=============>................] - ETA: 1s - loss: 1.5568 - accuracy: 0.4508
797/1563 [==============>...............] - ETA: 1s - loss: 1.5563 - accuracy: 0.4510
835/1563 [===============>..............] - ETA: 0s - loss: 1.5565 - accuracy: 0.4503
873/1563 [===============>..............] - ETA: 0s - loss: 1.5548 - accuracy: 0.4512
911/1563 [================>.............] - ETA: 0s - loss: 1.5553 - accuracy: 0.4512
948/1563 [=================>............] - ETA: 0s - loss: 1.5539 - accuracy: 0.4512
986/1563 [=================>............] - ETA: 0s - loss: 1.5546 - accuracy: 0.4507
1023/1563 [==================>...........] - ETA: 0s - loss: 1.5529 - accuracy: 0.4508
1061/1563 [===================>..........] - ETA: 0s - loss: 1.5519 - accuracy: 0.4513
1099/1563 [====================>.........] - ETA: 0s - loss: 1.5510 - accuracy: 0.4514
1137/1563 [====================>.........] - ETA: 0s - loss: 1.5520 - accuracy: 0.4513
1175/1563 [=====================>........] - ETA: 0s - loss: 1.5505 - accuracy: 0.4519
1213/1563 [======================>.......] - ETA: 0s - loss: 1.5502 - accuracy: 0.4520
1251/1563 [=======================>......] - ETA: 0s - loss: 1.5506 - accuracy: 0.4520
1289/1563 [=======================>......] - ETA: 0s - loss: 1.5502 - accuracy: 0.4525
1327/1563 [========================>.....] - ETA: 0s - loss: 1.5498 - accuracy: 0.4524
1364/1563 [=========================>....] - ETA: 0s - loss: 1.5491 - accuracy: 0.4526
1401/1563 [=========================>....] - ETA: 0s - loss: 1.5476 - accuracy: 0.4527
1439/1563 [==========================>...] - ETA: 0s - loss: 1.5459 - accuracy: 0.4537
1477/1563 [===========================>..] - ETA: 0s - loss: 1.5441 - accuracy: 0.4544
1515/1563 [============================>.] - ETA: 0s - loss: 1.5427 - accuracy: 0.4550
1552/1563 [============================>.] - ETA: 0s - loss: 1.5430 - accuracy: 0.4551
1563/1563 [==============================] - 2s 1ms/step - loss: 1.5425 - accuracy: 0.4553
Epoch 4/10
1/1563 [..............................] - ETA: 2s - loss: 1.5180 - accuracy: 0.5000
38/1563 [..............................] - ETA: 2s - loss: 1.5325 - accuracy: 0.4498
76/1563 [>.............................] - ETA: 2s - loss: 1.5137 - accuracy: 0.4638
113/1563 [=>............................] - ETA: 1s - loss: 1.5061 - accuracy: 0.4682
151/1563 [=>............................] - ETA: 1s - loss: 1.4969 - accuracy: 0.4688
189/1563 [==>...........................] - ETA: 1s - loss: 1.4880 - accuracy: 0.4727
227/1563 [===>..........................] - ETA: 1s - loss: 1.4953 - accuracy: 0.4701
265/1563 [====>.........................] - ETA: 1s - loss: 1.4991 - accuracy: 0.4702
303/1563 [====>.........................] - ETA: 1s - loss: 1.4904 - accuracy: 0.4743
341/1563 [=====>........................] - ETA: 1s - loss: 1.4917 - accuracy: 0.4709
379/1563 [======>.......................] - ETA: 1s - loss: 1.4924 - accuracy: 0.4730
417/1563 [=======>......................] - ETA: 1s - loss: 1.4941 - accuracy: 0.4729
455/1563 [=======>......................] - ETA: 1s - loss: 1.4916 - accuracy: 0.4739
493/1563 [========>.....................] - ETA: 1s - loss: 1.4877 - accuracy: 0.4757
531/1563 [=========>....................] - ETA: 1s - loss: 1.4881 - accuracy: 0.4743
569/1563 [=========>....................] - ETA: 1s - loss: 1.4915 - accuracy: 0.4733
607/1563 [==========>...................] - ETA: 1s - loss: 1.4888 - accuracy: 0.4738
645/1563 [===========>..................] - ETA: 1s - loss: 1.4873 - accuracy: 0.4748
683/1563 [============>.................] - ETA: 1s - loss: 1.4870 - accuracy: 0.4758
720/1563 [============>.................] - ETA: 1s - loss: 1.4864 - accuracy: 0.4767
757/1563 [=============>................] - ETA: 1s - loss: 1.4869 - accuracy: 0.4766
795/1563 [==============>...............] - ETA: 1s - loss: 1.4863 - accuracy: 0.4772
833/1563 [==============>...............] - ETA: 0s - loss: 1.4857 - accuracy: 0.4772
871/1563 [===============>..............] - ETA: 0s - loss: 1.4854 - accuracy: 0.4766
908/1563 [================>.............] - ETA: 0s - loss: 1.4868 - accuracy: 0.4769
946/1563 [=================>............] - ETA: 0s - loss: 1.4860 - accuracy: 0.4767
984/1563 [=================>............] - ETA: 0s - loss: 1.4866 - accuracy: 0.4768
1022/1563 [==================>...........] - ETA: 0s - loss: 1.4848 - accuracy: 0.4774
1060/1563 [===================>..........] - ETA: 0s - loss: 1.4836 - accuracy: 0.4774
1098/1563 [====================>.........] - ETA: 0s - loss: 1.4830 - accuracy: 0.4773
1136/1563 [====================>.........] - ETA: 0s - loss: 1.4820 - accuracy: 0.4776
1174/1563 [=====================>........] - ETA: 0s - loss: 1.4824 - accuracy: 0.4773
1211/1563 [======================>.......] - ETA: 0s - loss: 1.4814 - accuracy: 0.4774
1248/1563 [======================>.......] - ETA: 0s - loss: 1.4812 - accuracy: 0.4775
1285/1563 [=======================>......] - ETA: 0s - loss: 1.4814 - accuracy: 0.4770
1322/1563 [========================>.....] - ETA: 0s - loss: 1.4809 - accuracy: 0.4771
1360/1563 [=========================>....] - ETA: 0s - loss: 1.4821 - accuracy: 0.4766
1398/1563 [=========================>....] - ETA: 0s - loss: 1.4817 - accuracy: 0.4764
1436/1563 [==========================>...] - ETA: 0s - loss: 1.4814 - accuracy: 0.4767
1474/1563 [===========================>..] - ETA: 0s - loss: 1.4813 - accuracy: 0.4768
1511/1563 [============================>.] - ETA: 0s - loss: 1.4797 - accuracy: 0.4776
1548/1563 [============================>.] - ETA: 0s - loss: 1.4793 - accuracy: 0.4776
1563/1563 [==============================] - 2s 1ms/step - loss: 1.4790 - accuracy: 0.4775
Epoch 5/10
1/1563 [..............................] - ETA: 2s - loss: 1.4912 - accuracy: 0.4062
38/1563 [..............................] - ETA: 2s - loss: 1.4592 - accuracy: 0.4934
76/1563 [>.............................] - ETA: 2s - loss: 1.4478 - accuracy: 0.5045
114/1563 [=>............................] - ETA: 1s - loss: 1.4487 - accuracy: 0.5030
152/1563 [=>............................] - ETA: 1s - loss: 1.4472 - accuracy: 0.4992
190/1563 [==>...........................] - ETA: 1s - loss: 1.4515 - accuracy: 0.4944
228/1563 [===>..........................] - ETA: 1s - loss: 1.4479 - accuracy: 0.4931
265/1563 [====>.........................] - ETA: 1s - loss: 1.4440 - accuracy: 0.4973
303/1563 [====>.........................] - ETA: 1s - loss: 1.4404 - accuracy: 0.4988
341/1563 [=====>........................] - ETA: 1s - loss: 1.4441 - accuracy: 0.4984
379/1563 [======>.......................] - ETA: 1s - loss: 1.4432 - accuracy: 0.4979
417/1563 [=======>......................] - ETA: 1s - loss: 1.4435 - accuracy: 0.4973
455/1563 [=======>......................] - ETA: 1s - loss: 1.4422 - accuracy: 0.4972
493/1563 [========>.....................] - ETA: 1s - loss: 1.4433 - accuracy: 0.4975
531/1563 [=========>....................] - ETA: 1s - loss: 1.4436 - accuracy: 0.4976
569/1563 [=========>....................] - ETA: 1s - loss: 1.4441 - accuracy: 0.4974
606/1563 [==========>...................] - ETA: 1s - loss: 1.4467 - accuracy: 0.4968
644/1563 [===========>..................] - ETA: 1s - loss: 1.4481 - accuracy: 0.4966
682/1563 [============>.................] - ETA: 1s - loss: 1.4467 - accuracy: 0.4966
720/1563 [============>.................] - ETA: 1s - loss: 1.4444 - accuracy: 0.4972
758/1563 [=============>................] - ETA: 1s - loss: 1.4459 - accuracy: 0.4973
796/1563 [==============>...............] - ETA: 1s - loss: 1.4445 - accuracy: 0.4975
834/1563 [===============>..............] - ETA: 0s - loss: 1.4428 - accuracy: 0.4978
871/1563 [===============>..............] - ETA: 0s - loss: 1.4429 - accuracy: 0.4969
909/1563 [================>.............] - ETA: 0s - loss: 1.4430 - accuracy: 0.4960
946/1563 [=================>............] - ETA: 0s - loss: 1.4422 - accuracy: 0.4963
983/1563 [=================>............] - ETA: 0s - loss: 1.4413 - accuracy: 0.4964
1020/1563 [==================>...........] - ETA: 0s - loss: 1.4414 - accuracy: 0.4954
1058/1563 [===================>..........] - ETA: 0s - loss: 1.4411 - accuracy: 0.4959
1095/1563 [====================>.........] - ETA: 0s - loss: 1.4400 - accuracy: 0.4958
1133/1563 [====================>.........] - ETA: 0s - loss: 1.4404 - accuracy: 0.4961
1171/1563 [=====================>........] - ETA: 0s - loss: 1.4383 - accuracy: 0.4969
1209/1563 [======================>.......] - ETA: 0s - loss: 1.4352 - accuracy: 0.4974
1247/1563 [======================>.......] - ETA: 0s - loss: 1.4327 - accuracy: 0.4988
1285/1563 [=======================>......] - ETA: 0s - loss: 1.4324 - accuracy: 0.4992
1323/1563 [========================>.....] - ETA: 0s - loss: 1.4323 - accuracy: 0.4992
1361/1563 [=========================>....] - ETA: 0s - loss: 1.4316 - accuracy: 0.4996
1399/1563 [=========================>....] - ETA: 0s - loss: 1.4321 - accuracy: 0.4997
1437/1563 [==========================>...] - ETA: 0s - loss: 1.4332 - accuracy: 0.4991
1475/1563 [===========================>..] - ETA: 0s - loss: 1.4332 - accuracy: 0.4989
1513/1563 [============================>.] - ETA: 0s - loss: 1.4334 - accuracy: 0.4988
1550/1563 [============================>.] - ETA: 0s - loss: 1.4327 - accuracy: 0.4991
1563/1563 [==============================] - 2s 1ms/step - loss: 1.4322 - accuracy: 0.4992
Epoch 6/10
1/1563 [..............................] - ETA: 2s - loss: 1.3752 - accuracy: 0.5000
39/1563 [..............................] - ETA: 2s - loss: 1.4192 - accuracy: 0.5072
76/1563 [>.............................] - ETA: 2s - loss: 1.3923 - accuracy: 0.5189
114/1563 [=>............................] - ETA: 1s - loss: 1.4010 - accuracy: 0.5118
152/1563 [=>............................] - ETA: 1s - loss: 1.4048 - accuracy: 0.5058
190/1563 [==>...........................] - ETA: 1s - loss: 1.3998 - accuracy: 0.5071
228/1563 [===>..........................] - ETA: 1s - loss: 1.4031 - accuracy: 0.5052
266/1563 [====>.........................] - ETA: 1s - loss: 1.3971 - accuracy: 0.5075
303/1563 [====>.........................] - ETA: 1s - loss: 1.4001 - accuracy: 0.5059
341/1563 [=====>........................] - ETA: 1s - loss: 1.4038 - accuracy: 0.5052
379/1563 [======>.......................] - ETA: 1s - loss: 1.4036 - accuracy: 0.5052
417/1563 [=======>......................] - ETA: 1s - loss: 1.4059 - accuracy: 0.5052
455/1563 [=======>......................] - ETA: 1s - loss: 1.4048 - accuracy: 0.5058
493/1563 [========>.....................] - ETA: 1s - loss: 1.4019 - accuracy: 0.5072
530/1563 [=========>....................] - ETA: 1s - loss: 1.4024 - accuracy: 0.5070
568/1563 [=========>....................] - ETA: 1s - loss: 1.3991 - accuracy: 0.5074
606/1563 [==========>...................] - ETA: 1s - loss: 1.3995 - accuracy: 0.5064
643/1563 [===========>..................] - ETA: 1s - loss: 1.3975 - accuracy: 0.5080
680/1563 [============>.................] - ETA: 1s - loss: 1.3971 - accuracy: 0.5088
717/1563 [============>.................] - ETA: 1s - loss: 1.3987 - accuracy: 0.5075
755/1563 [=============>................] - ETA: 1s - loss: 1.3993 - accuracy: 0.5064
793/1563 [==============>...............] - ETA: 1s - loss: 1.3984 - accuracy: 0.5070
831/1563 [==============>...............] - ETA: 0s - loss: 1.3960 - accuracy: 0.5083
869/1563 [===============>..............] - ETA: 0s - loss: 1.3971 - accuracy: 0.5076
907/1563 [================>.............] - ETA: 0s - loss: 1.3956 - accuracy: 0.5078
944/1563 [=================>............] - ETA: 0s - loss: 1.3936 - accuracy: 0.5088
981/1563 [=================>............] - ETA: 0s - loss: 1.3950 - accuracy: 0.5088
1019/1563 [==================>...........] - ETA: 0s - loss: 1.3964 - accuracy: 0.5081
1057/1563 [===================>..........] - ETA: 0s - loss: 1.3955 - accuracy: 0.5080
1095/1563 [====================>.........] - ETA: 0s - loss: 1.3948 - accuracy: 0.5085
1133/1563 [====================>.........] - ETA: 0s - loss: 1.3944 - accuracy: 0.5084
1171/1563 [=====================>........] - ETA: 0s - loss: 1.3927 - accuracy: 0.5089
1208/1563 [======================>.......] - ETA: 0s - loss: 1.3917 - accuracy: 0.5092
1246/1563 [======================>.......] - ETA: 0s - loss: 1.3902 - accuracy: 0.5102
1283/1563 [=======================>......] - ETA: 0s - loss: 1.3902 - accuracy: 0.5105
1320/1563 [========================>.....] - ETA: 0s - loss: 1.3903 - accuracy: 0.5107
1357/1563 [=========================>....] - ETA: 0s - loss: 1.3892 - accuracy: 0.5111
1395/1563 [=========================>....] - ETA: 0s - loss: 1.3900 - accuracy: 0.5109
1433/1563 [==========================>...] - ETA: 0s - loss: 1.3896 - accuracy: 0.5111
1471/1563 [===========================>..] - ETA: 0s - loss: 1.3890 - accuracy: 0.5115
1508/1563 [===========================>..] - ETA: 0s - loss: 1.3882 - accuracy: 0.5120
1546/1563 [============================>.] - ETA: 0s - loss: 1.3883 - accuracy: 0.5118
1563/1563 [==============================] - 2s 1ms/step - loss: 1.3882 - accuracy: 0.5120
Epoch 7/10
1/1563 [..............................] - ETA: 2s - loss: 1.6378 - accuracy: 0.3750
39/1563 [..............................] - ETA: 2s - loss: 1.3917 - accuracy: 0.5192
77/1563 [>.............................] - ETA: 2s - loss: 1.3584 - accuracy: 0.5321
115/1563 [=>............................] - ETA: 1s - loss: 1.3611 - accuracy: 0.5272
153/1563 [=>............................] - ETA: 1s - loss: 1.3613 - accuracy: 0.5243
191/1563 [==>...........................] - ETA: 1s - loss: 1.3566 - accuracy: 0.5209
229/1563 [===>..........................] - ETA: 1s - loss: 1.3542 - accuracy: 0.5227
267/1563 [====>.........................] - ETA: 1s - loss: 1.3538 - accuracy: 0.5224
305/1563 [====>.........................] - ETA: 1s - loss: 1.3473 - accuracy: 0.5244
343/1563 [=====>........................] - ETA: 1s - loss: 1.3489 - accuracy: 0.5241
380/1563 [======>.......................] - ETA: 1s - loss: 1.3465 - accuracy: 0.5256
418/1563 [=======>......................] - ETA: 1s - loss: 1.3522 - accuracy: 0.5241
456/1563 [=======>......................] - ETA: 1s - loss: 1.3508 - accuracy: 0.5254
494/1563 [========>.....................] - ETA: 1s - loss: 1.3491 - accuracy: 0.5260
532/1563 [=========>....................] - ETA: 1s - loss: 1.3512 - accuracy: 0.5254
570/1563 [=========>....................] - ETA: 1s - loss: 1.3493 - accuracy: 0.5267
607/1563 [==========>...................] - ETA: 1s - loss: 1.3527 - accuracy: 0.5254
645/1563 [===========>..................] - ETA: 1s - loss: 1.3528 - accuracy: 0.5246
683/1563 [============>.................] - ETA: 1s - loss: 1.3519 - accuracy: 0.5251
721/1563 [============>.................] - ETA: 1s - loss: 1.3499 - accuracy: 0.5263
759/1563 [=============>................] - ETA: 1s - loss: 1.3512 - accuracy: 0.5270
797/1563 [==============>...............] - ETA: 1s - loss: 1.3493 - accuracy: 0.5277
835/1563 [===============>..............] - ETA: 0s - loss: 1.3483 - accuracy: 0.5283
872/1563 [===============>..............] - ETA: 0s - loss: 1.3503 - accuracy: 0.5276
909/1563 [================>.............] - ETA: 0s - loss: 1.3499 - accuracy: 0.5276
946/1563 [=================>............] - ETA: 0s - loss: 1.3499 - accuracy: 0.5278
984/1563 [=================>............] - ETA: 0s - loss: 1.3513 - accuracy: 0.5274
1021/1563 [==================>...........] - ETA: 0s - loss: 1.3515 - accuracy: 0.5267
1059/1563 [===================>..........] - ETA: 0s - loss: 1.3517 - accuracy: 0.5263
1097/1563 [====================>.........] - ETA: 0s - loss: 1.3511 - accuracy: 0.5268
1135/1563 [====================>.........] - ETA: 0s - loss: 1.3505 - accuracy: 0.5267
1173/1563 [=====================>........] - ETA: 0s - loss: 1.3516 - accuracy: 0.5269
1211/1563 [======================>.......] - ETA: 0s - loss: 1.3521 - accuracy: 0.5267
1248/1563 [======================>.......] - ETA: 0s - loss: 1.3519 - accuracy: 0.5267
1285/1563 [=======================>......] - ETA: 0s - loss: 1.3516 - accuracy: 0.5268
1322/1563 [========================>.....] - ETA: 0s - loss: 1.3496 - accuracy: 0.5276
1360/1563 [=========================>....] - ETA: 0s - loss: 1.3498 - accuracy: 0.5277
1398/1563 [=========================>....] - ETA: 0s - loss: 1.3501 - accuracy: 0.5273
1436/1563 [==========================>...] - ETA: 0s - loss: 1.3500 - accuracy: 0.5273
1473/1563 [===========================>..] - ETA: 0s - loss: 1.3505 - accuracy: 0.5269
1511/1563 [============================>.] - ETA: 0s - loss: 1.3511 - accuracy: 0.5267
1548/1563 [============================>.] - ETA: 0s - loss: 1.3517 - accuracy: 0.5263
1563/1563 [==============================] - 2s 1ms/step - loss: 1.3512 - accuracy: 0.5262
Epoch 8/10
1/1563 [..............................] - ETA: 2s - loss: 1.3023 - accuracy: 0.5312
39/1563 [..............................] - ETA: 2s - loss: 1.2603 - accuracy: 0.5769
77/1563 [>.............................] - ETA: 1s - loss: 1.2862 - accuracy: 0.5605
115/1563 [=>............................] - ETA: 1s - loss: 1.2893 - accuracy: 0.5527
153/1563 [=>............................] - ETA: 1s - loss: 1.2898 - accuracy: 0.5525
191/1563 [==>...........................] - ETA: 1s - loss: 1.2885 - accuracy: 0.5538
229/1563 [===>..........................] - ETA: 1s - loss: 1.2958 - accuracy: 0.5516
267/1563 [====>.........................] - ETA: 1s - loss: 1.2975 - accuracy: 0.5489
305/1563 [====>.........................] - ETA: 1s - loss: 1.3048 - accuracy: 0.5461
342/1563 [=====>........................] - ETA: 1s - loss: 1.3085 - accuracy: 0.5452
380/1563 [======>.......................] - ETA: 1s - loss: 1.3116 - accuracy: 0.5436
418/1563 [=======>......................] - ETA: 1s - loss: 1.3094 - accuracy: 0.5444
456/1563 [=======>......................] - ETA: 1s - loss: 1.3086 - accuracy: 0.5432
494/1563 [========>.....................] - ETA: 1s - loss: 1.3041 - accuracy: 0.5449
531/1563 [=========>....................] - ETA: 1s - loss: 1.3048 - accuracy: 0.5441
569/1563 [=========>....................] - ETA: 1s - loss: 1.3047 - accuracy: 0.5448
606/1563 [==========>...................] - ETA: 1s - loss: 1.3068 - accuracy: 0.5438
643/1563 [===========>..................] - ETA: 1s - loss: 1.3100 - accuracy: 0.5427
681/1563 [============>.................] - ETA: 1s - loss: 1.3086 - accuracy: 0.5439
719/1563 [============>.................] - ETA: 1s - loss: 1.3112 - accuracy: 0.5429
757/1563 [=============>................] - ETA: 1s - loss: 1.3101 - accuracy: 0.5442
795/1563 [==============>...............] - ETA: 1s - loss: 1.3099 - accuracy: 0.5436
832/1563 [==============>...............] - ETA: 0s - loss: 1.3116 - accuracy: 0.5427
870/1563 [===============>..............] - ETA: 0s - loss: 1.3116 - accuracy: 0.5421
907/1563 [================>.............] - ETA: 0s - loss: 1.3100 - accuracy: 0.5425
945/1563 [=================>............] - ETA: 0s - loss: 1.3110 - accuracy: 0.5424
983/1563 [=================>............] - ETA: 0s - loss: 1.3091 - accuracy: 0.5432
1021/1563 [==================>...........] - ETA: 0s - loss: 1.3111 - accuracy: 0.5426
1059/1563 [===================>..........] - ETA: 0s - loss: 1.3118 - accuracy: 0.5421
1097/1563 [====================>.........] - ETA: 0s - loss: 1.3121 - accuracy: 0.5418
1135/1563 [====================>.........] - ETA: 0s - loss: 1.3131 - accuracy: 0.5412
1172/1563 [=====================>........] - ETA: 0s - loss: 1.3129 - accuracy: 0.5408
1209/1563 [======================>.......] - ETA: 0s - loss: 1.3122 - accuracy: 0.5414
1246/1563 [======================>.......] - ETA: 0s - loss: 1.3119 - accuracy: 0.5415
1284/1563 [=======================>......] - ETA: 0s - loss: 1.3140 - accuracy: 0.5409
1322/1563 [========================>.....] - ETA: 0s - loss: 1.3139 - accuracy: 0.5409
1360/1563 [=========================>....] - ETA: 0s - loss: 1.3134 - accuracy: 0.5406
1398/1563 [=========================>....] - ETA: 0s - loss: 1.3138 - accuracy: 0.5401
1436/1563 [==========================>...] - ETA: 0s - loss: 1.3129 - accuracy: 0.5407
1474/1563 [===========================>..] - ETA: 0s - loss: 1.3122 - accuracy: 0.5409
1512/1563 [============================>.] - ETA: 0s - loss: 1.3133 - accuracy: 0.5406
1549/1563 [============================>.] - ETA: 0s - loss: 1.3136 - accuracy: 0.5406
1563/1563 [==============================] - 2s 1ms/step - loss: 1.3143 - accuracy: 0.5403
Epoch 9/10
1/1563 [..............................] - ETA: 2s - loss: 1.4493 - accuracy: 0.3750
39/1563 [..............................] - ETA: 2s - loss: 1.2836 - accuracy: 0.5457
77/1563 [>.............................] - ETA: 1s - loss: 1.3202 - accuracy: 0.5284
115/1563 [=>............................] - ETA: 1s - loss: 1.3063 - accuracy: 0.5367
153/1563 [=>............................] - ETA: 1s - loss: 1.3058 - accuracy: 0.5417
190/1563 [==>...........................] - ETA: 1s - loss: 1.3129 - accuracy: 0.5421
228/1563 [===>..........................] - ETA: 1s - loss: 1.3104 - accuracy: 0.5424
266/1563 [====>.........................] - ETA: 1s - loss: 1.3059 - accuracy: 0.5457
303/1563 [====>.........................] - ETA: 1s - loss: 1.3041 - accuracy: 0.5449
341/1563 [=====>........................] - ETA: 1s - loss: 1.3036 - accuracy: 0.5428
379/1563 [======>.......................] - ETA: 1s - loss: 1.3014 - accuracy: 0.5434
417/1563 [=======>......................] - ETA: 1s - loss: 1.2962 - accuracy: 0.5441
455/1563 [=======>......................] - ETA: 1s - loss: 1.2969 - accuracy: 0.5426
492/1563 [========>.....................] - ETA: 1s - loss: 1.2922 - accuracy: 0.5434
529/1563 [=========>....................] - ETA: 1s - loss: 1.2906 - accuracy: 0.5439
567/1563 [=========>....................] - ETA: 1s - loss: 1.2907 - accuracy: 0.5443
605/1563 [==========>...................] - ETA: 1s - loss: 1.2904 - accuracy: 0.5444
643/1563 [===========>..................] - ETA: 1s - loss: 1.2884 - accuracy: 0.5461
681/1563 [============>.................] - ETA: 1s - loss: 1.2901 - accuracy: 0.5462
719/1563 [============>.................] - ETA: 1s - loss: 1.2868 - accuracy: 0.5477
757/1563 [=============>................] - ETA: 1s - loss: 1.2879 - accuracy: 0.5473
795/1563 [==============>...............] - ETA: 1s - loss: 1.2875 - accuracy: 0.5474
833/1563 [==============>...............] - ETA: 0s - loss: 1.2872 - accuracy: 0.5481
870/1563 [===============>..............] - ETA: 0s - loss: 1.2814 - accuracy: 0.5497
907/1563 [================>.............] - ETA: 0s - loss: 1.2831 - accuracy: 0.5496
944/1563 [=================>............] - ETA: 0s - loss: 1.2825 - accuracy: 0.5493
982/1563 [=================>............] - ETA: 0s - loss: 1.2827 - accuracy: 0.5495
1020/1563 [==================>...........] - ETA: 0s - loss: 1.2853 - accuracy: 0.5487
1057/1563 [===================>..........] - ETA: 0s - loss: 1.2851 - accuracy: 0.5489
1095/1563 [====================>.........] - ETA: 0s - loss: 1.2855 - accuracy: 0.5486
1132/1563 [====================>.........] - ETA: 0s - loss: 1.2863 - accuracy: 0.5485
1170/1563 [=====================>........] - ETA: 0s - loss: 1.2845 - accuracy: 0.5495
1208/1563 [======================>.......] - ETA: 0s - loss: 1.2845 - accuracy: 0.5488
1246/1563 [======================>.......] - ETA: 0s - loss: 1.2854 - accuracy: 0.5479
1284/1563 [=======================>......] - ETA: 0s - loss: 1.2842 - accuracy: 0.5486
1322/1563 [========================>.....] - ETA: 0s - loss: 1.2834 - accuracy: 0.5491
1360/1563 [=========================>....] - ETA: 0s - loss: 1.2835 - accuracy: 0.5490
1398/1563 [=========================>....] - ETA: 0s - loss: 1.2834 - accuracy: 0.5488
1435/1563 [==========================>...] - ETA: 0s - loss: 1.2838 - accuracy: 0.5486
1472/1563 [===========================>..] - ETA: 0s - loss: 1.2834 - accuracy: 0.5488
1509/1563 [===========================>..] - ETA: 0s - loss: 1.2835 - accuracy: 0.5486
1546/1563 [============================>.] - ETA: 0s - loss: 1.2833 - accuracy: 0.5488
1563/1563 [==============================] - 2s 1ms/step - loss: 1.2829 - accuracy: 0.5495
Epoch 10/10
1/1563 [..............................] - ETA: 2s - loss: 1.0790 - accuracy: 0.6875
39/1563 [..............................] - ETA: 2s - loss: 1.2489 - accuracy: 0.5833
77/1563 [>.............................] - ETA: 2s - loss: 1.2359 - accuracy: 0.5747
115/1563 [=>............................] - ETA: 1s - loss: 1.2613 - accuracy: 0.5622
152/1563 [=>............................] - ETA: 1s - loss: 1.2601 - accuracy: 0.5646
189/1563 [==>...........................] - ETA: 1s - loss: 1.2623 - accuracy: 0.5618
226/1563 [===>..........................] - ETA: 1s - loss: 1.2618 - accuracy: 0.5601
263/1563 [====>.........................] - ETA: 1s - loss: 1.2632 - accuracy: 0.5618
299/1563 [====>.........................] - ETA: 1s - loss: 1.2524 - accuracy: 0.5656
336/1563 [=====>........................] - ETA: 1s - loss: 1.2533 - accuracy: 0.5652
373/1563 [======>.......................] - ETA: 1s - loss: 1.2496 - accuracy: 0.5653
410/1563 [======>.......................] - ETA: 1s - loss: 1.2490 - accuracy: 0.5659
447/1563 [=======>......................] - ETA: 1s - loss: 1.2489 - accuracy: 0.5657
484/1563 [========>.....................] - ETA: 1s - loss: 1.2514 - accuracy: 0.5641
521/1563 [=========>....................] - ETA: 1s - loss: 1.2495 - accuracy: 0.5647
558/1563 [=========>....................] - ETA: 1s - loss: 1.2515 - accuracy: 0.5638
595/1563 [==========>...................] - ETA: 1s - loss: 1.2511 - accuracy: 0.5645
632/1563 [===========>..................] - ETA: 1s - loss: 1.2493 - accuracy: 0.5655
669/1563 [===========>..................] - ETA: 1s - loss: 1.2505 - accuracy: 0.5645
706/1563 [============>.................] - ETA: 1s - loss: 1.2522 - accuracy: 0.5635
743/1563 [=============>................] - ETA: 1s - loss: 1.2541 - accuracy: 0.5621
781/1563 [=============>................] - ETA: 1s - loss: 1.2530 - accuracy: 0.5618
819/1563 [==============>...............] - ETA: 1s - loss: 1.2520 - accuracy: 0.5622
857/1563 [===============>..............] - ETA: 0s - loss: 1.2535 - accuracy: 0.5616
895/1563 [================>.............] - ETA: 0s - loss: 1.2537 - accuracy: 0.5614
932/1563 [================>.............] - ETA: 0s - loss: 1.2547 - accuracy: 0.5608
969/1563 [=================>............] - ETA: 0s - loss: 1.2565 - accuracy: 0.5602
1006/1563 [==================>...........] - ETA: 0s - loss: 1.2562 - accuracy: 0.5603
1043/1563 [===================>..........] - ETA: 0s - loss: 1.2551 - accuracy: 0.5603
1080/1563 [===================>..........] - ETA: 0s - loss: 1.2560 - accuracy: 0.5600
1118/1563 [====================>.........] - ETA: 0s - loss: 1.2568 - accuracy: 0.5603
1156/1563 [=====================>........] - ETA: 0s - loss: 1.2566 - accuracy: 0.5601
1194/1563 [=====================>........] - ETA: 0s - loss: 1.2576 - accuracy: 0.5598
1231/1563 [======================>.......] - ETA: 0s - loss: 1.2586 - accuracy: 0.5596
1268/1563 [=======================>......] - ETA: 0s - loss: 1.2569 - accuracy: 0.5601
1305/1563 [========================>.....] - ETA: 0s - loss: 1.2560 - accuracy: 0.5602
1342/1563 [========================>.....] - ETA: 0s - loss: 1.2544 - accuracy: 0.5608
1377/1563 [=========================>....] - ETA: 0s - loss: 1.2544 - accuracy: 0.5607
1413/1563 [==========================>...] - ETA: 0s - loss: 1.2541 - accuracy: 0.5611
1451/1563 [==========================>...] - ETA: 0s - loss: 1.2539 - accuracy: 0.5613
1489/1563 [===========================>..] - ETA: 0s - loss: 1.2538 - accuracy: 0.5612
1526/1563 [============================>.] - ETA: 0s - loss: 1.2538 - accuracy: 0.5614
1563/1563 [==============================] - ETA: 0s - loss: 1.2542 - accuracy: 0.5610
1563/1563 [==============================] - 2s 1ms/step - loss: 1.2542 - accuracy: 0.5610
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
flatten (Flatten) (None, 3072) 0
dense (Dense) (None, 3000) 9219000
dense_1 (Dense) (None, 1000) 3001000
dense_2 (Dense) (None, 10) 10010
=================================================================
Total params: 12,230,010
Trainable params: 12,230,010
Non-trainable params: 0
_________________________________________________________________
None
wall time seconds (without loading test data set)= 37.201841831207275
hydro04
[SSH] completed
[SSH] exit-status: 0
[workspace] $ /bin/sh -xe /tmp/jenkins7755366998763078675.sh
+ scp 'HYDRO_REMOTE:~svchydrojenkins/python/tensorflow_time.txt' /var/lib/jenkins/jobs/tensorflow/workspace
Recording plot data
Saving plot series data from: /var/lib/jenkins/jobs/tensorflow/workspace/tensorflow_time.txt
Finished: SUCCESS
![[Jenkins]](/static/755cd4af/images/svgs/logo.svg){kind=logo}
