Started by timer
Running as SYSTEM
Building in workspace /var/lib/jenkins/jobs/tensorflow/workspace
[SSH] script:
module load anaconda3_gpu
module load cudnn
cd python
rm -f tensorflow_test.txt
srun --nodes=1 --ntasks-per-node=1 --mem=16g \
--time=00:05:00 \
--partition=a100rome \
--account=bbmb-hydro \
--gpus-per-node=1 \
--gpu-bind=closest \
python3 cifar10gpu.py | tee tensorflow_test.txt
YVALUE=`grep "wall time seconds" tensorflow_test.txt | cut -d'=' -f2`
echo YVALUE=$YVALUE > tensorflow_time.txt
[SSH] executing...
srun: job 98986 queued and waiting for resources
srun: job 98986 has been allocated resources
2024-12-02 07:43:02.929873: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-12-02 07:43:03.073253: E tensorflow/stream_executor/cuda/cuda_blas.cc:2981] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-12-02 07:43:04.070692: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /sw/spack/hydrogpu-2022-06/apps/cudnn/8.4.0.27-11.6-gcc-11.3.0-qbhsfav/lib:/sw/spack/hydrogpu-2022-06/apps/cuda/11.7.0-gcc-11.3.0-3ysno6b/lib64:/sw/spack/hydrogpu-2022-06/apps/openmpi/4.1.4-gcc-11.3.0-i6koipj/lib:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib64:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib
2024-12-02 07:43:04.070765: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /sw/spack/hydrogpu-2022-06/apps/cudnn/8.4.0.27-11.6-gcc-11.3.0-qbhsfav/lib:/sw/spack/hydrogpu-2022-06/apps/cuda/11.7.0-gcc-11.3.0-3ysno6b/lib64:/sw/spack/hydrogpu-2022-06/apps/openmpi/4.1.4-gcc-11.3.0-i6koipj/lib:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib64:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib
2024-12-02 07:43:04.070771: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
2024-12-02 07:43:07.183870: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-12-02 07:43:07.555410: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1616] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 79148 MB memory: -> device: 0, name: NVIDIA A100 80GB PCIe, pci bus id: 0000:21:00.0, compute capability: 8.0
(50000, 32, 32, 3) (10000, 32, 32, 3)
Epoch 1/10
2024-12-02 07:43:10.245987: I tensorflow/stream_executor/cuda/cuda_blas.cc:1614] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
1/1563 [..............................] - ETA: 28:20 - loss: 2.4187 - accuracy: 0.0625
36/1563 [..............................] - ETA: 2s - loss: 2.2740 - accuracy: 0.1510
72/1563 [>.............................] - ETA: 2s - loss: 2.1964 - accuracy: 0.1901
109/1563 [=>............................] - ETA: 2s - loss: 2.1414 - accuracy: 0.2139
146/1563 [=>............................] - ETA: 1s - loss: 2.1086 - accuracy: 0.2318
183/1563 [==>...........................] - ETA: 1s - loss: 2.0829 - accuracy: 0.2469
220/1563 [===>..........................] - ETA: 1s - loss: 2.0543 - accuracy: 0.2553
257/1563 [===>..........................] - ETA: 1s - loss: 2.0374 - accuracy: 0.2611
295/1563 [====>.........................] - ETA: 1s - loss: 2.0206 - accuracy: 0.2671
332/1563 [=====>........................] - ETA: 1s - loss: 2.0055 - accuracy: 0.2747
370/1563 [======>.......................] - ETA: 1s - loss: 1.9915 - accuracy: 0.2793
408/1563 [======>.......................] - ETA: 1s - loss: 1.9762 - accuracy: 0.2849
445/1563 [=======>......................] - ETA: 1s - loss: 1.9661 - accuracy: 0.2892
482/1563 [========>.....................] - ETA: 1s - loss: 1.9589 - accuracy: 0.2931
520/1563 [========>.....................] - ETA: 1s - loss: 1.9500 - accuracy: 0.2973
558/1563 [=========>....................] - ETA: 1s - loss: 1.9415 - accuracy: 0.3023
596/1563 [==========>...................] - ETA: 1s - loss: 1.9331 - accuracy: 0.3053
634/1563 [===========>..................] - ETA: 1s - loss: 1.9245 - accuracy: 0.3086
672/1563 [===========>..................] - ETA: 1s - loss: 1.9183 - accuracy: 0.3118
710/1563 [============>.................] - ETA: 1s - loss: 1.9127 - accuracy: 0.3149
748/1563 [=============>................] - ETA: 1s - loss: 1.9047 - accuracy: 0.3186
786/1563 [==============>...............] - ETA: 1s - loss: 1.8981 - accuracy: 0.3216
824/1563 [==============>...............] - ETA: 1s - loss: 1.8944 - accuracy: 0.3230
862/1563 [===============>..............] - ETA: 0s - loss: 1.8887 - accuracy: 0.3252
900/1563 [================>.............] - ETA: 0s - loss: 1.8848 - accuracy: 0.3264
938/1563 [=================>............] - ETA: 0s - loss: 1.8780 - accuracy: 0.3292
976/1563 [=================>............] - ETA: 0s - loss: 1.8731 - accuracy: 0.3306
1014/1563 [==================>...........] - ETA: 0s - loss: 1.8674 - accuracy: 0.3323
1052/1563 [===================>..........] - ETA: 0s - loss: 1.8617 - accuracy: 0.3354
1090/1563 [===================>..........] - ETA: 0s - loss: 1.8572 - accuracy: 0.3372
1128/1563 [====================>.........] - ETA: 0s - loss: 1.8523 - accuracy: 0.3393
1166/1563 [=====================>........] - ETA: 0s - loss: 1.8484 - accuracy: 0.3411
1203/1563 [======================>.......] - ETA: 0s - loss: 1.8441 - accuracy: 0.3429
1241/1563 [======================>.......] - ETA: 0s - loss: 1.8402 - accuracy: 0.3439
1279/1563 [=======================>......] - ETA: 0s - loss: 1.8368 - accuracy: 0.3450
1317/1563 [========================>.....] - ETA: 0s - loss: 1.8335 - accuracy: 0.3459
1355/1563 [=========================>....] - ETA: 0s - loss: 1.8299 - accuracy: 0.3474
1393/1563 [=========================>....] - ETA: 0s - loss: 1.8271 - accuracy: 0.3485
1431/1563 [==========================>...] - ETA: 0s - loss: 1.8246 - accuracy: 0.3494
1469/1563 [===========================>..] - ETA: 0s - loss: 1.8210 - accuracy: 0.3512
1507/1563 [===========================>..] - ETA: 0s - loss: 1.8179 - accuracy: 0.3524
1544/1563 [============================>.] - ETA: 0s - loss: 1.8157 - accuracy: 0.3533
1563/1563 [==============================] - 3s 1ms/step - loss: 1.8139 - accuracy: 0.3540
Epoch 2/10
1/1563 [..............................] - ETA: 2s - loss: 1.6694 - accuracy: 0.5000
38/1563 [..............................] - ETA: 2s - loss: 1.7148 - accuracy: 0.3939
76/1563 [>.............................] - ETA: 2s - loss: 1.6911 - accuracy: 0.4034
114/1563 [=>............................] - ETA: 1s - loss: 1.6803 - accuracy: 0.4084
152/1563 [=>............................] - ETA: 1s - loss: 1.6733 - accuracy: 0.4102
190/1563 [==>...........................] - ETA: 1s - loss: 1.6713 - accuracy: 0.4130
228/1563 [===>..........................] - ETA: 1s - loss: 1.6676 - accuracy: 0.4134
262/1563 [====>.........................] - ETA: 1s - loss: 1.6606 - accuracy: 0.4167
300/1563 [====>.........................] - ETA: 1s - loss: 1.6619 - accuracy: 0.4155
338/1563 [=====>........................] - ETA: 1s - loss: 1.6607 - accuracy: 0.4162
376/1563 [======>.......................] - ETA: 1s - loss: 1.6578 - accuracy: 0.4179
414/1563 [======>.......................] - ETA: 1s - loss: 1.6544 - accuracy: 0.4189
452/1563 [=======>......................] - ETA: 1s - loss: 1.6517 - accuracy: 0.4195
490/1563 [========>.....................] - ETA: 1s - loss: 1.6514 - accuracy: 0.4188
528/1563 [=========>....................] - ETA: 1s - loss: 1.6518 - accuracy: 0.4176
565/1563 [=========>....................] - ETA: 1s - loss: 1.6495 - accuracy: 0.4183
603/1563 [==========>...................] - ETA: 1s - loss: 1.6444 - accuracy: 0.4184
641/1563 [===========>..................] - ETA: 1s - loss: 1.6432 - accuracy: 0.4194
678/1563 [============>.................] - ETA: 1s - loss: 1.6405 - accuracy: 0.4203
716/1563 [============>.................] - ETA: 1s - loss: 1.6405 - accuracy: 0.4205
754/1563 [=============>................] - ETA: 1s - loss: 1.6374 - accuracy: 0.4215
792/1563 [==============>...............] - ETA: 1s - loss: 1.6360 - accuracy: 0.4219
830/1563 [==============>...............] - ETA: 0s - loss: 1.6374 - accuracy: 0.4219
868/1563 [===============>..............] - ETA: 0s - loss: 1.6375 - accuracy: 0.4219
906/1563 [================>.............] - ETA: 0s - loss: 1.6374 - accuracy: 0.4219
944/1563 [=================>............] - ETA: 0s - loss: 1.6369 - accuracy: 0.4221
982/1563 [=================>............] - ETA: 0s - loss: 1.6352 - accuracy: 0.4230
1019/1563 [==================>...........] - ETA: 0s - loss: 1.6334 - accuracy: 0.4239
1057/1563 [===================>..........] - ETA: 0s - loss: 1.6308 - accuracy: 0.4248
1095/1563 [====================>.........] - ETA: 0s - loss: 1.6289 - accuracy: 0.4251
1133/1563 [====================>.........] - ETA: 0s - loss: 1.6276 - accuracy: 0.4263
1171/1563 [=====================>........] - ETA: 0s - loss: 1.6283 - accuracy: 0.4264
1208/1563 [======================>.......] - ETA: 0s - loss: 1.6277 - accuracy: 0.4272
1245/1563 [======================>.......] - ETA: 0s - loss: 1.6275 - accuracy: 0.4274
1282/1563 [=======================>......] - ETA: 0s - loss: 1.6259 - accuracy: 0.4278
1319/1563 [========================>.....] - ETA: 0s - loss: 1.6254 - accuracy: 0.4280
1357/1563 [=========================>....] - ETA: 0s - loss: 1.6246 - accuracy: 0.4280
1394/1563 [=========================>....] - ETA: 0s - loss: 1.6236 - accuracy: 0.4280
1431/1563 [==========================>...] - ETA: 0s - loss: 1.6237 - accuracy: 0.4281
1468/1563 [===========================>..] - ETA: 0s - loss: 1.6231 - accuracy: 0.4285
1505/1563 [===========================>..] - ETA: 0s - loss: 1.6229 - accuracy: 0.4283
1542/1563 [============================>.] - ETA: 0s - loss: 1.6232 - accuracy: 0.4273
1563/1563 [==============================] - 2s 1ms/step - loss: 1.6223 - accuracy: 0.4278
Epoch 3/10
1/1563 [..............................] - ETA: 2s - loss: 1.5167 - accuracy: 0.5000
38/1563 [..............................] - ETA: 2s - loss: 1.5552 - accuracy: 0.4564
76/1563 [>.............................] - ETA: 2s - loss: 1.5546 - accuracy: 0.4576
113/1563 [=>............................] - ETA: 1s - loss: 1.5437 - accuracy: 0.4616
151/1563 [=>............................] - ETA: 1s - loss: 1.5427 - accuracy: 0.4572
189/1563 [==>...........................] - ETA: 1s - loss: 1.5515 - accuracy: 0.4544
227/1563 [===>..........................] - ETA: 1s - loss: 1.5545 - accuracy: 0.4521
265/1563 [====>.........................] - ETA: 1s - loss: 1.5606 - accuracy: 0.4489
303/1563 [====>.........................] - ETA: 1s - loss: 1.5609 - accuracy: 0.4483
341/1563 [=====>........................] - ETA: 1s - loss: 1.5624 - accuracy: 0.4486
379/1563 [======>.......................] - ETA: 1s - loss: 1.5579 - accuracy: 0.4500
417/1563 [=======>......................] - ETA: 1s - loss: 1.5576 - accuracy: 0.4496
454/1563 [=======>......................] - ETA: 1s - loss: 1.5561 - accuracy: 0.4501
492/1563 [========>.....................] - ETA: 1s - loss: 1.5544 - accuracy: 0.4497
530/1563 [=========>....................] - ETA: 1s - loss: 1.5548 - accuracy: 0.4491
568/1563 [=========>....................] - ETA: 1s - loss: 1.5535 - accuracy: 0.4494
606/1563 [==========>...................] - ETA: 1s - loss: 1.5515 - accuracy: 0.4496
644/1563 [===========>..................] - ETA: 1s - loss: 1.5510 - accuracy: 0.4503
682/1563 [============>.................] - ETA: 1s - loss: 1.5514 - accuracy: 0.4503
720/1563 [============>.................] - ETA: 1s - loss: 1.5500 - accuracy: 0.4521
757/1563 [=============>................] - ETA: 1s - loss: 1.5500 - accuracy: 0.4518
795/1563 [==============>...............] - ETA: 1s - loss: 1.5518 - accuracy: 0.4506
833/1563 [==============>...............] - ETA: 0s - loss: 1.5527 - accuracy: 0.4514
871/1563 [===============>..............] - ETA: 0s - loss: 1.5537 - accuracy: 0.4512
909/1563 [================>.............] - ETA: 0s - loss: 1.5506 - accuracy: 0.4524
947/1563 [=================>............] - ETA: 0s - loss: 1.5486 - accuracy: 0.4534
985/1563 [=================>............] - ETA: 0s - loss: 1.5477 - accuracy: 0.4540
1023/1563 [==================>...........] - ETA: 0s - loss: 1.5448 - accuracy: 0.4553
1061/1563 [===================>..........] - ETA: 0s - loss: 1.5461 - accuracy: 0.4549
1099/1563 [====================>.........] - ETA: 0s - loss: 1.5449 - accuracy: 0.4552
1137/1563 [====================>.........] - ETA: 0s - loss: 1.5465 - accuracy: 0.4549
1175/1563 [=====================>........] - ETA: 0s - loss: 1.5462 - accuracy: 0.4544
1213/1563 [======================>.......] - ETA: 0s - loss: 1.5465 - accuracy: 0.4538
1251/1563 [=======================>......] - ETA: 0s - loss: 1.5470 - accuracy: 0.4539
1286/1563 [=======================>......] - ETA: 0s - loss: 1.5464 - accuracy: 0.4544
1324/1563 [========================>.....] - ETA: 0s - loss: 1.5452 - accuracy: 0.4554
1362/1563 [=========================>....] - ETA: 0s - loss: 1.5445 - accuracy: 0.4555
1400/1563 [=========================>....] - ETA: 0s - loss: 1.5426 - accuracy: 0.4564
1438/1563 [==========================>...] - ETA: 0s - loss: 1.5426 - accuracy: 0.4569
1476/1563 [===========================>..] - ETA: 0s - loss: 1.5427 - accuracy: 0.4568
1514/1563 [============================>.] - ETA: 0s - loss: 1.5416 - accuracy: 0.4569
1551/1563 [============================>.] - ETA: 0s - loss: 1.5399 - accuracy: 0.4574
1563/1563 [==============================] - 2s 1ms/step - loss: 1.5403 - accuracy: 0.4574
Epoch 4/10
1/1563 [..............................] - ETA: 2s - loss: 1.5438 - accuracy: 0.3438
36/1563 [..............................] - ETA: 2s - loss: 1.4720 - accuracy: 0.4566
74/1563 [>.............................] - ETA: 2s - loss: 1.4931 - accuracy: 0.4624
112/1563 [=>............................] - ETA: 1s - loss: 1.4840 - accuracy: 0.4718
150/1563 [=>............................] - ETA: 1s - loss: 1.4985 - accuracy: 0.4715
188/1563 [==>...........................] - ETA: 1s - loss: 1.4936 - accuracy: 0.4737
226/1563 [===>..........................] - ETA: 1s - loss: 1.4855 - accuracy: 0.4773
264/1563 [====>.........................] - ETA: 1s - loss: 1.4879 - accuracy: 0.4743
302/1563 [====>.........................] - ETA: 1s - loss: 1.4857 - accuracy: 0.4759
339/1563 [=====>........................] - ETA: 1s - loss: 1.4794 - accuracy: 0.4787
377/1563 [======>.......................] - ETA: 1s - loss: 1.4842 - accuracy: 0.4765
415/1563 [======>.......................] - ETA: 1s - loss: 1.4816 - accuracy: 0.4782
453/1563 [=======>......................] - ETA: 1s - loss: 1.4845 - accuracy: 0.4770
491/1563 [========>.....................] - ETA: 1s - loss: 1.4874 - accuracy: 0.4770
529/1563 [=========>....................] - ETA: 1s - loss: 1.4881 - accuracy: 0.4754
567/1563 [=========>....................] - ETA: 1s - loss: 1.4896 - accuracy: 0.4750
605/1563 [==========>...................] - ETA: 1s - loss: 1.4873 - accuracy: 0.4747
643/1563 [===========>..................] - ETA: 1s - loss: 1.4869 - accuracy: 0.4737
681/1563 [============>.................] - ETA: 1s - loss: 1.4892 - accuracy: 0.4724
719/1563 [============>.................] - ETA: 1s - loss: 1.4877 - accuracy: 0.4734
757/1563 [=============>................] - ETA: 1s - loss: 1.4860 - accuracy: 0.4740
795/1563 [==============>...............] - ETA: 1s - loss: 1.4862 - accuracy: 0.4739
833/1563 [==============>...............] - ETA: 0s - loss: 1.4845 - accuracy: 0.4745
871/1563 [===============>..............] - ETA: 0s - loss: 1.4852 - accuracy: 0.4738
908/1563 [================>.............] - ETA: 0s - loss: 1.4853 - accuracy: 0.4737
946/1563 [=================>............] - ETA: 0s - loss: 1.4844 - accuracy: 0.4745
984/1563 [=================>............] - ETA: 0s - loss: 1.4843 - accuracy: 0.4749
1022/1563 [==================>...........] - ETA: 0s - loss: 1.4841 - accuracy: 0.4746
1057/1563 [===================>..........] - ETA: 0s - loss: 1.4842 - accuracy: 0.4747
1095/1563 [====================>.........] - ETA: 0s - loss: 1.4848 - accuracy: 0.4740
1133/1563 [====================>.........] - ETA: 0s - loss: 1.4843 - accuracy: 0.4738
1171/1563 [=====================>........] - ETA: 0s - loss: 1.4839 - accuracy: 0.4745
1209/1563 [======================>.......] - ETA: 0s - loss: 1.4829 - accuracy: 0.4749
1246/1563 [======================>.......] - ETA: 0s - loss: 1.4819 - accuracy: 0.4755
1284/1563 [=======================>......] - ETA: 0s - loss: 1.4811 - accuracy: 0.4755
1322/1563 [========================>.....] - ETA: 0s - loss: 1.4804 - accuracy: 0.4756
1360/1563 [=========================>....] - ETA: 0s - loss: 1.4806 - accuracy: 0.4757
1398/1563 [=========================>....] - ETA: 0s - loss: 1.4818 - accuracy: 0.4755
1436/1563 [==========================>...] - ETA: 0s - loss: 1.4819 - accuracy: 0.4756
1474/1563 [===========================>..] - ETA: 0s - loss: 1.4815 - accuracy: 0.4756
1511/1563 [============================>.] - ETA: 0s - loss: 1.4816 - accuracy: 0.4756
1549/1563 [============================>.] - ETA: 0s - loss: 1.4813 - accuracy: 0.4756
1563/1563 [==============================] - 2s 1ms/step - loss: 1.4819 - accuracy: 0.4752
Epoch 5/10
1/1563 [..............................] - ETA: 2s - loss: 1.6401 - accuracy: 0.3750
39/1563 [..............................] - ETA: 2s - loss: 1.4557 - accuracy: 0.4824
77/1563 [>.............................] - ETA: 1s - loss: 1.4410 - accuracy: 0.4931
115/1563 [=>............................] - ETA: 1s - loss: 1.4541 - accuracy: 0.4864
153/1563 [=>............................] - ETA: 1s - loss: 1.4548 - accuracy: 0.4867
191/1563 [==>...........................] - ETA: 1s - loss: 1.4582 - accuracy: 0.4846
229/1563 [===>..........................] - ETA: 1s - loss: 1.4683 - accuracy: 0.4797
267/1563 [====>.........................] - ETA: 1s - loss: 1.4644 - accuracy: 0.4799
302/1563 [====>.........................] - ETA: 1s - loss: 1.4587 - accuracy: 0.4829
340/1563 [=====>........................] - ETA: 1s - loss: 1.4594 - accuracy: 0.4834
378/1563 [======>.......................] - ETA: 1s - loss: 1.4572 - accuracy: 0.4839
416/1563 [======>.......................] - ETA: 1s - loss: 1.4545 - accuracy: 0.4831
454/1563 [=======>......................] - ETA: 1s - loss: 1.4540 - accuracy: 0.4839
492/1563 [========>.....................] - ETA: 1s - loss: 1.4546 - accuracy: 0.4832
530/1563 [=========>....................] - ETA: 1s - loss: 1.4487 - accuracy: 0.4857
567/1563 [=========>....................] - ETA: 1s - loss: 1.4471 - accuracy: 0.4857
605/1563 [==========>...................] - ETA: 1s - loss: 1.4441 - accuracy: 0.4865
643/1563 [===========>..................] - ETA: 1s - loss: 1.4432 - accuracy: 0.4864
681/1563 [============>.................] - ETA: 1s - loss: 1.4416 - accuracy: 0.4862
719/1563 [============>.................] - ETA: 1s - loss: 1.4405 - accuracy: 0.4873
757/1563 [=============>................] - ETA: 1s - loss: 1.4398 - accuracy: 0.4872
795/1563 [==============>...............] - ETA: 1s - loss: 1.4413 - accuracy: 0.4876
833/1563 [==============>...............] - ETA: 0s - loss: 1.4404 - accuracy: 0.4884
871/1563 [===============>..............] - ETA: 0s - loss: 1.4409 - accuracy: 0.4882
909/1563 [================>.............] - ETA: 0s - loss: 1.4394 - accuracy: 0.4891
946/1563 [=================>............] - ETA: 0s - loss: 1.4402 - accuracy: 0.4884
984/1563 [=================>............] - ETA: 0s - loss: 1.4411 - accuracy: 0.4888
1022/1563 [==================>...........] - ETA: 0s - loss: 1.4397 - accuracy: 0.4899
1060/1563 [===================>..........] - ETA: 0s - loss: 1.4401 - accuracy: 0.4898
1099/1563 [====================>.........] - ETA: 0s - loss: 1.4395 - accuracy: 0.4899
1137/1563 [====================>.........] - ETA: 0s - loss: 1.4383 - accuracy: 0.4904
1175/1563 [=====================>........] - ETA: 0s - loss: 1.4372 - accuracy: 0.4908
1213/1563 [======================>.......] - ETA: 0s - loss: 1.4362 - accuracy: 0.4919
1251/1563 [=======================>......] - ETA: 0s - loss: 1.4360 - accuracy: 0.4921
1288/1563 [=======================>......] - ETA: 0s - loss: 1.4349 - accuracy: 0.4925
1326/1563 [========================>.....] - ETA: 0s - loss: 1.4342 - accuracy: 0.4927
1362/1563 [=========================>....] - ETA: 0s - loss: 1.4343 - accuracy: 0.4929
1400/1563 [=========================>....] - ETA: 0s - loss: 1.4342 - accuracy: 0.4926
1439/1563 [==========================>...] - ETA: 0s - loss: 1.4344 - accuracy: 0.4924
1477/1563 [===========================>..] - ETA: 0s - loss: 1.4341 - accuracy: 0.4926
1515/1563 [============================>.] - ETA: 0s - loss: 1.4333 - accuracy: 0.4923
1553/1563 [============================>.] - ETA: 0s - loss: 1.4330 - accuracy: 0.4926
1563/1563 [==============================] - 2s 1ms/step - loss: 1.4326 - accuracy: 0.4927
Epoch 6/10
1/1563 [..............................] - ETA: 2s - loss: 1.3868 - accuracy: 0.5312
39/1563 [..............................] - ETA: 2s - loss: 1.4050 - accuracy: 0.5152
77/1563 [>.............................] - ETA: 1s - loss: 1.4021 - accuracy: 0.5130
115/1563 [=>............................] - ETA: 1s - loss: 1.4012 - accuracy: 0.5136
153/1563 [=>............................] - ETA: 1s - loss: 1.4048 - accuracy: 0.5123
191/1563 [==>...........................] - ETA: 1s - loss: 1.3956 - accuracy: 0.5137
229/1563 [===>..........................] - ETA: 1s - loss: 1.3912 - accuracy: 0.5142
267/1563 [====>.........................] - ETA: 1s - loss: 1.3862 - accuracy: 0.5144
304/1563 [====>.........................] - ETA: 1s - loss: 1.3900 - accuracy: 0.5121
342/1563 [=====>........................] - ETA: 1s - loss: 1.3843 - accuracy: 0.5148
380/1563 [======>.......................] - ETA: 1s - loss: 1.3868 - accuracy: 0.5137
418/1563 [=======>......................] - ETA: 1s - loss: 1.3878 - accuracy: 0.5143
456/1563 [=======>......................] - ETA: 1s - loss: 1.3924 - accuracy: 0.5119
494/1563 [========>.....................] - ETA: 1s - loss: 1.3950 - accuracy: 0.5130
532/1563 [=========>....................] - ETA: 1s - loss: 1.3956 - accuracy: 0.5127
570/1563 [=========>....................] - ETA: 1s - loss: 1.3950 - accuracy: 0.5132
608/1563 [==========>...................] - ETA: 1s - loss: 1.3972 - accuracy: 0.5125
646/1563 [===========>..................] - ETA: 1s - loss: 1.3970 - accuracy: 0.5121
684/1563 [============>.................] - ETA: 1s - loss: 1.3962 - accuracy: 0.5127
722/1563 [============>.................] - ETA: 1s - loss: 1.3964 - accuracy: 0.5135
760/1563 [=============>................] - ETA: 1s - loss: 1.3980 - accuracy: 0.5118
798/1563 [==============>...............] - ETA: 1s - loss: 1.3982 - accuracy: 0.5122
836/1563 [===============>..............] - ETA: 0s - loss: 1.3980 - accuracy: 0.5114
874/1563 [===============>..............] - ETA: 0s - loss: 1.3987 - accuracy: 0.5111
912/1563 [================>.............] - ETA: 0s - loss: 1.3985 - accuracy: 0.5113
950/1563 [=================>............] - ETA: 0s - loss: 1.3993 - accuracy: 0.5104
988/1563 [=================>............] - ETA: 0s - loss: 1.3989 - accuracy: 0.5109
1026/1563 [==================>...........] - ETA: 0s - loss: 1.3968 - accuracy: 0.5119
1064/1563 [===================>..........] - ETA: 0s - loss: 1.3969 - accuracy: 0.5120
1102/1563 [====================>.........] - ETA: 0s - loss: 1.3969 - accuracy: 0.5121
1140/1563 [====================>.........] - ETA: 0s - loss: 1.3952 - accuracy: 0.5127
1178/1563 [=====================>........] - ETA: 0s - loss: 1.3936 - accuracy: 0.5132
1215/1563 [======================>.......] - ETA: 0s - loss: 1.3937 - accuracy: 0.5130
1253/1563 [=======================>......] - ETA: 0s - loss: 1.3925 - accuracy: 0.5136
1291/1563 [=======================>......] - ETA: 0s - loss: 1.3924 - accuracy: 0.5136
1329/1563 [========================>.....] - ETA: 0s - loss: 1.3914 - accuracy: 0.5142
1367/1563 [=========================>....] - ETA: 0s - loss: 1.3906 - accuracy: 0.5140
1405/1563 [=========================>....] - ETA: 0s - loss: 1.3896 - accuracy: 0.5139
1443/1563 [==========================>...] - ETA: 0s - loss: 1.3906 - accuracy: 0.5129
1481/1563 [===========================>..] - ETA: 0s - loss: 1.3906 - accuracy: 0.5133
1519/1563 [============================>.] - ETA: 0s - loss: 1.3904 - accuracy: 0.5136
1557/1563 [============================>.] - ETA: 0s - loss: 1.3895 - accuracy: 0.5139
1563/1563 [==============================] - 2s 1ms/step - loss: 1.3892 - accuracy: 0.5140
Epoch 7/10
1/1563 [..............................] - ETA: 2s - loss: 1.0715 - accuracy: 0.5625
39/1563 [..............................] - ETA: 2s - loss: 1.3698 - accuracy: 0.5441
77/1563 [>.............................] - ETA: 2s - loss: 1.3869 - accuracy: 0.5183
115/1563 [=>............................] - ETA: 1s - loss: 1.3663 - accuracy: 0.5207
153/1563 [=>............................] - ETA: 1s - loss: 1.3707 - accuracy: 0.5229
191/1563 [==>...........................] - ETA: 1s - loss: 1.3762 - accuracy: 0.5167
229/1563 [===>..........................] - ETA: 1s - loss: 1.3805 - accuracy: 0.5145
266/1563 [====>.........................] - ETA: 1s - loss: 1.3728 - accuracy: 0.5166
303/1563 [====>.........................] - ETA: 1s - loss: 1.3696 - accuracy: 0.5184
341/1563 [=====>........................] - ETA: 1s - loss: 1.3716 - accuracy: 0.5181
379/1563 [======>.......................] - ETA: 1s - loss: 1.3663 - accuracy: 0.5217
417/1563 [=======>......................] - ETA: 1s - loss: 1.3652 - accuracy: 0.5222
455/1563 [=======>......................] - ETA: 1s - loss: 1.3635 - accuracy: 0.5227
493/1563 [========>.....................] - ETA: 1s - loss: 1.3639 - accuracy: 0.5206
531/1563 [=========>....................] - ETA: 1s - loss: 1.3628 - accuracy: 0.5224
569/1563 [=========>....................] - ETA: 1s - loss: 1.3609 - accuracy: 0.5224
606/1563 [==========>...................] - ETA: 1s - loss: 1.3600 - accuracy: 0.5237
644/1563 [===========>..................] - ETA: 1s - loss: 1.3601 - accuracy: 0.5236
682/1563 [============>.................] - ETA: 1s - loss: 1.3610 - accuracy: 0.5233
720/1563 [============>.................] - ETA: 1s - loss: 1.3605 - accuracy: 0.5230
758/1563 [=============>................] - ETA: 1s - loss: 1.3578 - accuracy: 0.5235
796/1563 [==============>...............] - ETA: 1s - loss: 1.3573 - accuracy: 0.5245
834/1563 [===============>..............] - ETA: 0s - loss: 1.3579 - accuracy: 0.5250
872/1563 [===============>..............] - ETA: 0s - loss: 1.3574 - accuracy: 0.5249
911/1563 [================>.............] - ETA: 0s - loss: 1.3570 - accuracy: 0.5251
949/1563 [=================>............] - ETA: 0s - loss: 1.3588 - accuracy: 0.5251
987/1563 [=================>............] - ETA: 0s - loss: 1.3576 - accuracy: 0.5256
1025/1563 [==================>...........] - ETA: 0s - loss: 1.3569 - accuracy: 0.5255
1063/1563 [===================>..........] - ETA: 0s - loss: 1.3573 - accuracy: 0.5252
1101/1563 [====================>.........] - ETA: 0s - loss: 1.3559 - accuracy: 0.5255
1139/1563 [====================>.........] - ETA: 0s - loss: 1.3558 - accuracy: 0.5253
1177/1563 [=====================>........] - ETA: 0s - loss: 1.3550 - accuracy: 0.5256
1215/1563 [======================>.......] - ETA: 0s - loss: 1.3546 - accuracy: 0.5262
1252/1563 [=======================>......] - ETA: 0s - loss: 1.3537 - accuracy: 0.5265
1290/1563 [=======================>......] - ETA: 0s - loss: 1.3538 - accuracy: 0.5262
1328/1563 [========================>.....] - ETA: 0s - loss: 1.3531 - accuracy: 0.5266
1366/1563 [=========================>....] - ETA: 0s - loss: 1.3523 - accuracy: 0.5269
1404/1563 [=========================>....] - ETA: 0s - loss: 1.3520 - accuracy: 0.5271
1442/1563 [==========================>...] - ETA: 0s - loss: 1.3511 - accuracy: 0.5273
1479/1563 [===========================>..] - ETA: 0s - loss: 1.3515 - accuracy: 0.5270
1517/1563 [============================>.] - ETA: 0s - loss: 1.3505 - accuracy: 0.5277
1555/1563 [============================>.] - ETA: 0s - loss: 1.3504 - accuracy: 0.5279
1563/1563 [==============================] - 2s 1ms/step - loss: 1.3507 - accuracy: 0.5278
Epoch 8/10
1/1563 [..............................] - ETA: 2s - loss: 1.2187 - accuracy: 0.6250
38/1563 [..............................] - ETA: 2s - loss: 1.3328 - accuracy: 0.5493
76/1563 [>.............................] - ETA: 2s - loss: 1.2996 - accuracy: 0.5502
114/1563 [=>............................] - ETA: 1s - loss: 1.2970 - accuracy: 0.5504
152/1563 [=>............................] - ETA: 1s - loss: 1.3044 - accuracy: 0.5434
190/1563 [==>...........................] - ETA: 1s - loss: 1.3045 - accuracy: 0.5400
227/1563 [===>..........................] - ETA: 1s - loss: 1.3107 - accuracy: 0.5384
264/1563 [====>.........................] - ETA: 1s - loss: 1.3157 - accuracy: 0.5359
302/1563 [====>.........................] - ETA: 1s - loss: 1.3122 - accuracy: 0.5384
340/1563 [=====>........................] - ETA: 1s - loss: 1.3161 - accuracy: 0.5361
378/1563 [======>.......................] - ETA: 1s - loss: 1.3187 - accuracy: 0.5340
416/1563 [======>.......................] - ETA: 1s - loss: 1.3149 - accuracy: 0.5347
454/1563 [=======>......................] - ETA: 1s - loss: 1.3111 - accuracy: 0.5372
492/1563 [========>.....................] - ETA: 1s - loss: 1.3111 - accuracy: 0.5377
530/1563 [=========>....................] - ETA: 1s - loss: 1.3137 - accuracy: 0.5369
568/1563 [=========>....................] - ETA: 1s - loss: 1.3133 - accuracy: 0.5380
606/1563 [==========>...................] - ETA: 1s - loss: 1.3092 - accuracy: 0.5408
644/1563 [===========>..................] - ETA: 1s - loss: 1.3106 - accuracy: 0.5396
682/1563 [============>.................] - ETA: 1s - loss: 1.3094 - accuracy: 0.5400
720/1563 [============>.................] - ETA: 1s - loss: 1.3074 - accuracy: 0.5405
758/1563 [=============>................] - ETA: 1s - loss: 1.3092 - accuracy: 0.5402
796/1563 [==============>...............] - ETA: 1s - loss: 1.3092 - accuracy: 0.5402
834/1563 [===============>..............] - ETA: 0s - loss: 1.3102 - accuracy: 0.5397
872/1563 [===============>..............] - ETA: 0s - loss: 1.3101 - accuracy: 0.5396
910/1563 [================>.............] - ETA: 0s - loss: 1.3083 - accuracy: 0.5404
948/1563 [=================>............] - ETA: 0s - loss: 1.3105 - accuracy: 0.5395
986/1563 [=================>............] - ETA: 0s - loss: 1.3101 - accuracy: 0.5395
1024/1563 [==================>...........] - ETA: 0s - loss: 1.3110 - accuracy: 0.5392
1062/1563 [===================>..........] - ETA: 0s - loss: 1.3127 - accuracy: 0.5384
1099/1563 [====================>.........] - ETA: 0s - loss: 1.3134 - accuracy: 0.5380
1137/1563 [====================>.........] - ETA: 0s - loss: 1.3149 - accuracy: 0.5377
1175/1563 [=====================>........] - ETA: 0s - loss: 1.3139 - accuracy: 0.5375
1213/1563 [======================>.......] - ETA: 0s - loss: 1.3149 - accuracy: 0.5374
1251/1563 [=======================>......] - ETA: 0s - loss: 1.3151 - accuracy: 0.5371
1289/1563 [=======================>......] - ETA: 0s - loss: 1.3152 - accuracy: 0.5368
1327/1563 [========================>.....] - ETA: 0s - loss: 1.3157 - accuracy: 0.5365
1365/1563 [=========================>....] - ETA: 0s - loss: 1.3160 - accuracy: 0.5367
1403/1563 [=========================>....] - ETA: 0s - loss: 1.3176 - accuracy: 0.5361
1441/1563 [==========================>...] - ETA: 0s - loss: 1.3167 - accuracy: 0.5363
1479/1563 [===========================>..] - ETA: 0s - loss: 1.3154 - accuracy: 0.5372
1517/1563 [============================>.] - ETA: 0s - loss: 1.3152 - accuracy: 0.5375
1556/1563 [============================>.] - ETA: 0s - loss: 1.3148 - accuracy: 0.5375
1563/1563 [==============================] - 2s 1ms/step - loss: 1.3151 - accuracy: 0.5375
Epoch 9/10
1/1563 [..............................] - ETA: 2s - loss: 1.3538 - accuracy: 0.5000
39/1563 [..............................] - ETA: 2s - loss: 1.2640 - accuracy: 0.5753
77/1563 [>.............................] - ETA: 1s - loss: 1.2819 - accuracy: 0.5576
115/1563 [=>............................] - ETA: 1s - loss: 1.2705 - accuracy: 0.5633
152/1563 [=>............................] - ETA: 1s - loss: 1.2769 - accuracy: 0.5586
190/1563 [==>...........................] - ETA: 1s - loss: 1.2657 - accuracy: 0.5630
227/1563 [===>..........................] - ETA: 1s - loss: 1.2703 - accuracy: 0.5558
265/1563 [====>.........................] - ETA: 1s - loss: 1.2668 - accuracy: 0.5560
303/1563 [====>.........................] - ETA: 1s - loss: 1.2675 - accuracy: 0.5595
341/1563 [=====>........................] - ETA: 1s - loss: 1.2755 - accuracy: 0.5556
379/1563 [======>.......................] - ETA: 1s - loss: 1.2773 - accuracy: 0.5572
417/1563 [=======>......................] - ETA: 1s - loss: 1.2775 - accuracy: 0.5570
455/1563 [=======>......................] - ETA: 1s - loss: 1.2809 - accuracy: 0.5549
492/1563 [========>.....................] - ETA: 1s - loss: 1.2815 - accuracy: 0.5551
530/1563 [=========>....................] - ETA: 1s - loss: 1.2850 - accuracy: 0.5540
568/1563 [=========>....................] - ETA: 1s - loss: 1.2864 - accuracy: 0.5524
606/1563 [==========>...................] - ETA: 1s - loss: 1.2905 - accuracy: 0.5511
644/1563 [===========>..................] - ETA: 1s - loss: 1.2880 - accuracy: 0.5516
682/1563 [============>.................] - ETA: 1s - loss: 1.2857 - accuracy: 0.5529
720/1563 [============>.................] - ETA: 1s - loss: 1.2871 - accuracy: 0.5532
757/1563 [=============>................] - ETA: 1s - loss: 1.2863 - accuracy: 0.5539
795/1563 [==============>...............] - ETA: 1s - loss: 1.2871 - accuracy: 0.5527
833/1563 [==============>...............] - ETA: 0s - loss: 1.2866 - accuracy: 0.5523
871/1563 [===============>..............] - ETA: 0s - loss: 1.2863 - accuracy: 0.5524
909/1563 [================>.............] - ETA: 0s - loss: 1.2869 - accuracy: 0.5516
947/1563 [=================>............] - ETA: 0s - loss: 1.2885 - accuracy: 0.5509
985/1563 [=================>............] - ETA: 0s - loss: 1.2886 - accuracy: 0.5506
1023/1563 [==================>...........] - ETA: 0s - loss: 1.2880 - accuracy: 0.5504
1061/1563 [===================>..........] - ETA: 0s - loss: 1.2888 - accuracy: 0.5500
1099/1563 [====================>.........] - ETA: 0s - loss: 1.2888 - accuracy: 0.5503
1137/1563 [====================>.........] - ETA: 0s - loss: 1.2877 - accuracy: 0.5499
1175/1563 [=====================>........] - ETA: 0s - loss: 1.2853 - accuracy: 0.5503
1213/1563 [======================>.......] - ETA: 0s - loss: 1.2852 - accuracy: 0.5502
1251/1563 [=======================>......] - ETA: 0s - loss: 1.2841 - accuracy: 0.5507
1289/1563 [=======================>......] - ETA: 0s - loss: 1.2855 - accuracy: 0.5502
1327/1563 [========================>.....] - ETA: 0s - loss: 1.2853 - accuracy: 0.5501
1365/1563 [=========================>....] - ETA: 0s - loss: 1.2855 - accuracy: 0.5497
1403/1563 [=========================>....] - ETA: 0s - loss: 1.2851 - accuracy: 0.5502
1441/1563 [==========================>...] - ETA: 0s - loss: 1.2856 - accuracy: 0.5501
1479/1563 [===========================>..] - ETA: 0s - loss: 1.2853 - accuracy: 0.5498
1517/1563 [============================>.] - ETA: 0s - loss: 1.2856 - accuracy: 0.5499
1555/1563 [============================>.] - ETA: 0s - loss: 1.2853 - accuracy: 0.5504
1563/1563 [==============================] - 2s 1ms/step - loss: 1.2856 - accuracy: 0.5504
Epoch 10/10
1/1563 [..............................] - ETA: 2s - loss: 1.5753 - accuracy: 0.4688
39/1563 [..............................] - ETA: 2s - loss: 1.2406 - accuracy: 0.5673
77/1563 [>.............................] - ETA: 1s - loss: 1.2333 - accuracy: 0.5743
115/1563 [=>............................] - ETA: 1s - loss: 1.2473 - accuracy: 0.5663
153/1563 [=>............................] - ETA: 1s - loss: 1.2421 - accuracy: 0.5694
191/1563 [==>...........................] - ETA: 1s - loss: 1.2545 - accuracy: 0.5636
229/1563 [===>..........................] - ETA: 1s - loss: 1.2508 - accuracy: 0.5652
267/1563 [====>.........................] - ETA: 1s - loss: 1.2517 - accuracy: 0.5662
305/1563 [====>.........................] - ETA: 1s - loss: 1.2533 - accuracy: 0.5641
343/1563 [=====>........................] - ETA: 1s - loss: 1.2555 - accuracy: 0.5640
381/1563 [======>.......................] - ETA: 1s - loss: 1.2562 - accuracy: 0.5632
419/1563 [=======>......................] - ETA: 1s - loss: 1.2584 - accuracy: 0.5626
457/1563 [=======>......................] - ETA: 1s - loss: 1.2579 - accuracy: 0.5650
495/1563 [========>.....................] - ETA: 1s - loss: 1.2581 - accuracy: 0.5652
533/1563 [=========>....................] - ETA: 1s - loss: 1.2558 - accuracy: 0.5654
571/1563 [=========>....................] - ETA: 1s - loss: 1.2527 - accuracy: 0.5669
609/1563 [==========>...................] - ETA: 1s - loss: 1.2511 - accuracy: 0.5665
647/1563 [===========>..................] - ETA: 1s - loss: 1.2518 - accuracy: 0.5660
685/1563 [============>.................] - ETA: 1s - loss: 1.2544 - accuracy: 0.5635
723/1563 [============>.................] - ETA: 1s - loss: 1.2539 - accuracy: 0.5635
761/1563 [=============>................] - ETA: 1s - loss: 1.2530 - accuracy: 0.5637
799/1563 [==============>...............] - ETA: 1s - loss: 1.2515 - accuracy: 0.5641
837/1563 [===============>..............] - ETA: 0s - loss: 1.2518 - accuracy: 0.5643
875/1563 [===============>..............] - ETA: 0s - loss: 1.2520 - accuracy: 0.5643
913/1563 [================>.............] - ETA: 0s - loss: 1.2524 - accuracy: 0.5640
951/1563 [=================>............] - ETA: 0s - loss: 1.2517 - accuracy: 0.5646
989/1563 [=================>............] - ETA: 0s - loss: 1.2520 - accuracy: 0.5642
1027/1563 [==================>...........] - ETA: 0s - loss: 1.2520 - accuracy: 0.5648
1065/1563 [===================>..........] - ETA: 0s - loss: 1.2514 - accuracy: 0.5646
1103/1563 [====================>.........] - ETA: 0s - loss: 1.2520 - accuracy: 0.5644
1141/1563 [====================>.........] - ETA: 0s - loss: 1.2524 - accuracy: 0.5645
1179/1563 [=====================>........] - ETA: 0s - loss: 1.2534 - accuracy: 0.5640
1217/1563 [======================>.......] - ETA: 0s - loss: 1.2516 - accuracy: 0.5646
1255/1563 [=======================>......] - ETA: 0s - loss: 1.2534 - accuracy: 0.5633
1293/1563 [=======================>......] - ETA: 0s - loss: 1.2530 - accuracy: 0.5633
1331/1563 [========================>.....] - ETA: 0s - loss: 1.2530 - accuracy: 0.5628
1369/1563 [=========================>....] - ETA: 0s - loss: 1.2534 - accuracy: 0.5625
1407/1563 [==========================>...] - ETA: 0s - loss: 1.2544 - accuracy: 0.5623
1445/1563 [==========================>...] - ETA: 0s - loss: 1.2537 - accuracy: 0.5623
1483/1563 [===========================>..] - ETA: 0s - loss: 1.2533 - accuracy: 0.5618
1521/1563 [============================>.] - ETA: 0s - loss: 1.2532 - accuracy: 0.5620
1558/1563 [============================>.] - ETA: 0s - loss: 1.2533 - accuracy: 0.5617
1563/1563 [==============================] - 2s 1ms/step - loss: 1.2531 - accuracy: 0.5619
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
flatten (Flatten) (None, 3072) 0
dense (Dense) (None, 3000) 9219000
dense_1 (Dense) (None, 1000) 3001000
dense_2 (Dense) (None, 10) 10010
=================================================================
Total params: 12,230,010
Trainable params: 12,230,010
Non-trainable params: 0
_________________________________________________________________
None
wall time seconds (without loading test data set)= 24.42150115966797
hydro03
[SSH] completed
[SSH] exit-status: 0
[workspace] $ /bin/sh -xe /tmp/jenkins8446579409734841768.sh
+ scp 'HYDRO_REMOTE:~svchydrojenkins/python/tensorflow_time.txt' /var/lib/jenkins/jobs/tensorflow/workspace
Recording plot data
Saving plot series data from: /var/lib/jenkins/jobs/tensorflow/workspace/tensorflow_time.txt
Finished: SUCCESS
![[Jenkins]](/static/755cd4af/images/svgs/logo.svg){kind=logo}
