Started by timer
Running as SYSTEM
Building in workspace /var/lib/jenkins/jobs/tensorflow/workspace
[SSH] script:
module load anaconda3_gpu
module load cudnn
cd python
rm -f tensorflow_test.txt
srun --nodes=1 --ntasks-per-node=1 --mem=16g \
--time=00:05:00 \
--partition=a100rome \
--account=bbmb-hydro \
--gpus-per-node=1 \
--gpu-bind=closest \
python3 cifar10gpu.py | tee tensorflow_test.txt
YVALUE=`grep "wall time seconds" tensorflow_test.txt | cut -d'=' -f2`
echo YVALUE=$YVALUE > tensorflow_time.txt
[SSH] executing...
srun: job 96453 queued and waiting for resources
srun: job 96453 has been allocated resources
2024-11-03 07:43:09.551988: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-11-03 07:43:51.661861: E tensorflow/stream_executor/cuda/cuda_blas.cc:2981] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-11-03 07:44:40.289945: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /sw/spack/hydrogpu-2022-06/apps/cudnn/8.4.0.27-11.6-gcc-11.3.0-qbhsfav/lib:/sw/spack/hydrogpu-2022-06/apps/cuda/11.7.0-gcc-11.3.0-3ysno6b/lib64:/sw/spack/hydrogpu-2022-06/apps/openmpi/4.1.4-gcc-11.3.0-i6koipj/lib:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib64:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib
2024-11-03 07:44:40.290801: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /sw/spack/hydrogpu-2022-06/apps/cudnn/8.4.0.27-11.6-gcc-11.3.0-qbhsfav/lib:/sw/spack/hydrogpu-2022-06/apps/cuda/11.7.0-gcc-11.3.0-3ysno6b/lib64:/sw/spack/hydrogpu-2022-06/apps/openmpi/4.1.4-gcc-11.3.0-i6koipj/lib:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib64:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib
2024-11-03 07:44:40.290811: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
2024-11-03 07:45:59.324882: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-11-03 07:46:03.982839: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1616] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 79148 MB memory: -> device: 0, name: NVIDIA A100 80GB PCIe, pci bus id: 0000:21:00.0, compute capability: 8.0
(50000, 32, 32, 3) (10000, 32, 32, 3)
Epoch 1/10
2024-11-03 07:46:15.887697: I tensorflow/stream_executor/cuda/cuda_blas.cc:1614] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
1/1563 [..............................] - ETA: 2:28:28 - loss: 2.3118 - accuracy: 0.0938
37/1563 [..............................] - ETA: 2s - loss: 2.2478 - accuracy: 0.1630
74/1563 [>.............................] - ETA: 2s - loss: 2.1828 - accuracy: 0.1913
112/1563 [=>............................] - ETA: 1s - loss: 2.1315 - accuracy: 0.2137
150/1563 [=>............................] - ETA: 1s - loss: 2.0962 - accuracy: 0.2265
188/1563 [==>...........................] - ETA: 1s - loss: 2.0733 - accuracy: 0.2357
226/1563 [===>..........................] - ETA: 1s - loss: 2.0494 - accuracy: 0.2474
264/1563 [====>.........................] - ETA: 1s - loss: 2.0287 - accuracy: 0.2582
303/1563 [====>.........................] - ETA: 1s - loss: 2.0156 - accuracy: 0.2638
342/1563 [=====>........................] - ETA: 1s - loss: 2.0010 - accuracy: 0.2726
381/1563 [======>.......................] - ETA: 1s - loss: 1.9873 - accuracy: 0.2771
419/1563 [=======>......................] - ETA: 1s - loss: 1.9789 - accuracy: 0.2800
457/1563 [=======>......................] - ETA: 1s - loss: 1.9687 - accuracy: 0.2855
495/1563 [========>.....................] - ETA: 1s - loss: 1.9555 - accuracy: 0.2902
533/1563 [=========>....................] - ETA: 1s - loss: 1.9458 - accuracy: 0.2943
571/1563 [=========>....................] - ETA: 1s - loss: 1.9356 - accuracy: 0.2987
610/1563 [==========>...................] - ETA: 1s - loss: 1.9289 - accuracy: 0.3022
650/1563 [===========>..................] - ETA: 1s - loss: 1.9223 - accuracy: 0.3051
689/1563 [============>.................] - ETA: 1s - loss: 1.9167 - accuracy: 0.3085
728/1563 [============>.................] - ETA: 1s - loss: 1.9106 - accuracy: 0.3107
766/1563 [=============>................] - ETA: 1s - loss: 1.9045 - accuracy: 0.3136
804/1563 [==============>...............] - ETA: 1s - loss: 1.8977 - accuracy: 0.3170
842/1563 [===============>..............] - ETA: 0s - loss: 1.8926 - accuracy: 0.3198
880/1563 [===============>..............] - ETA: 0s - loss: 1.8866 - accuracy: 0.3219
918/1563 [================>.............] - ETA: 0s - loss: 1.8812 - accuracy: 0.3242
955/1563 [=================>............] - ETA: 0s - loss: 1.8760 - accuracy: 0.3262
993/1563 [==================>...........] - ETA: 0s - loss: 1.8717 - accuracy: 0.3283
1031/1563 [==================>...........] - ETA: 0s - loss: 1.8665 - accuracy: 0.3300
1069/1563 [===================>..........] - ETA: 0s - loss: 1.8606 - accuracy: 0.3317
1107/1563 [====================>.........] - ETA: 0s - loss: 1.8566 - accuracy: 0.3331
1145/1563 [====================>.........] - ETA: 0s - loss: 1.8519 - accuracy: 0.3351
1183/1563 [=====================>........] - ETA: 0s - loss: 1.8470 - accuracy: 0.3372
1221/1563 [======================>.......] - ETA: 0s - loss: 1.8437 - accuracy: 0.3383
1259/1563 [=======================>......] - ETA: 0s - loss: 1.8397 - accuracy: 0.3403
1297/1563 [=======================>......] - ETA: 0s - loss: 1.8350 - accuracy: 0.3420
1335/1563 [========================>.....] - ETA: 0s - loss: 1.8313 - accuracy: 0.3434
1373/1563 [=========================>....] - ETA: 0s - loss: 1.8276 - accuracy: 0.3453
1411/1563 [==========================>...] - ETA: 0s - loss: 1.8244 - accuracy: 0.3467
1449/1563 [==========================>...] - ETA: 0s - loss: 1.8209 - accuracy: 0.3483
1487/1563 [===========================>..] - ETA: 0s - loss: 1.8173 - accuracy: 0.3496
1525/1563 [============================>.] - ETA: 0s - loss: 1.8140 - accuracy: 0.3508
1562/1563 [============================>.] - ETA: 0s - loss: 1.8118 - accuracy: 0.3522
1563/1563 [==============================] - 8s 1ms/step - loss: 1.8118 - accuracy: 0.3522
Epoch 2/10
1/1563 [..............................] - ETA: 2s - loss: 1.4996 - accuracy: 0.4375
39/1563 [..............................] - ETA: 2s - loss: 1.6728 - accuracy: 0.4087
77/1563 [>.............................] - ETA: 1s - loss: 1.6634 - accuracy: 0.4058
115/1563 [=>............................] - ETA: 1s - loss: 1.6515 - accuracy: 0.4079
153/1563 [=>............................] - ETA: 1s - loss: 1.6498 - accuracy: 0.4118
191/1563 [==>...........................] - ETA: 1s - loss: 1.6479 - accuracy: 0.4115
229/1563 [===>..........................] - ETA: 1s - loss: 1.6494 - accuracy: 0.4117
267/1563 [====>.........................] - ETA: 1s - loss: 1.6494 - accuracy: 0.4140
305/1563 [====>.........................] - ETA: 1s - loss: 1.6494 - accuracy: 0.4155
343/1563 [=====>........................] - ETA: 1s - loss: 1.6471 - accuracy: 0.4155
381/1563 [======>.......................] - ETA: 1s - loss: 1.6486 - accuracy: 0.4145
419/1563 [=======>......................] - ETA: 1s - loss: 1.6524 - accuracy: 0.4110
457/1563 [=======>......................] - ETA: 1s - loss: 1.6523 - accuracy: 0.4121
495/1563 [========>.....................] - ETA: 1s - loss: 1.6516 - accuracy: 0.4129
533/1563 [=========>....................] - ETA: 1s - loss: 1.6485 - accuracy: 0.4154
571/1563 [=========>....................] - ETA: 1s - loss: 1.6497 - accuracy: 0.4158
609/1563 [==========>...................] - ETA: 1s - loss: 1.6487 - accuracy: 0.4170
647/1563 [===========>..................] - ETA: 1s - loss: 1.6475 - accuracy: 0.4174
685/1563 [============>.................] - ETA: 1s - loss: 1.6482 - accuracy: 0.4172
723/1563 [============>.................] - ETA: 1s - loss: 1.6472 - accuracy: 0.4181
761/1563 [=============>................] - ETA: 1s - loss: 1.6457 - accuracy: 0.4189
799/1563 [==============>...............] - ETA: 1s - loss: 1.6461 - accuracy: 0.4184
837/1563 [===============>..............] - ETA: 0s - loss: 1.6458 - accuracy: 0.4185
875/1563 [===============>..............] - ETA: 0s - loss: 1.6452 - accuracy: 0.4181
912/1563 [================>.............] - ETA: 0s - loss: 1.6431 - accuracy: 0.4188
950/1563 [=================>............] - ETA: 0s - loss: 1.6438 - accuracy: 0.4187
988/1563 [=================>............] - ETA: 0s - loss: 1.6431 - accuracy: 0.4188
1026/1563 [==================>...........] - ETA: 0s - loss: 1.6427 - accuracy: 0.4189
1064/1563 [===================>..........] - ETA: 0s - loss: 1.6431 - accuracy: 0.4191
1102/1563 [====================>.........] - ETA: 0s - loss: 1.6420 - accuracy: 0.4195
1140/1563 [====================>.........] - ETA: 0s - loss: 1.6395 - accuracy: 0.4198
1178/1563 [=====================>........] - ETA: 0s - loss: 1.6369 - accuracy: 0.4213
1216/1563 [======================>.......] - ETA: 0s - loss: 1.6345 - accuracy: 0.4224
1254/1563 [=======================>......] - ETA: 0s - loss: 1.6347 - accuracy: 0.4224
1292/1563 [=======================>......] - ETA: 0s - loss: 1.6324 - accuracy: 0.4234
1330/1563 [========================>.....] - ETA: 0s - loss: 1.6313 - accuracy: 0.4235
1368/1563 [=========================>....] - ETA: 0s - loss: 1.6289 - accuracy: 0.4247
1406/1563 [=========================>....] - ETA: 0s - loss: 1.6265 - accuracy: 0.4258
1444/1563 [==========================>...] - ETA: 0s - loss: 1.6250 - accuracy: 0.4265
1482/1563 [===========================>..] - ETA: 0s - loss: 1.6246 - accuracy: 0.4262
1520/1563 [============================>.] - ETA: 0s - loss: 1.6241 - accuracy: 0.4259
1558/1563 [============================>.] - ETA: 0s - loss: 1.6240 - accuracy: 0.4263
1563/1563 [==============================] - 2s 1ms/step - loss: 1.6240 - accuracy: 0.4262
Epoch 3/10
1/1563 [..............................] - ETA: 2s - loss: 1.7228 - accuracy: 0.4062
39/1563 [..............................] - ETA: 2s - loss: 1.5727 - accuracy: 0.4327
77/1563 [>.............................] - ETA: 1s - loss: 1.5642 - accuracy: 0.4375
115/1563 [=>............................] - ETA: 1s - loss: 1.5549 - accuracy: 0.4443
153/1563 [=>............................] - ETA: 1s - loss: 1.5556 - accuracy: 0.4465
191/1563 [==>...........................] - ETA: 1s - loss: 1.5473 - accuracy: 0.4496
229/1563 [===>..........................] - ETA: 1s - loss: 1.5511 - accuracy: 0.4498
267/1563 [====>.........................] - ETA: 1s - loss: 1.5466 - accuracy: 0.4524
305/1563 [====>.........................] - ETA: 1s - loss: 1.5557 - accuracy: 0.4499
343/1563 [=====>........................] - ETA: 1s - loss: 1.5537 - accuracy: 0.4507
381/1563 [======>.......................] - ETA: 1s - loss: 1.5514 - accuracy: 0.4516
419/1563 [=======>......................] - ETA: 1s - loss: 1.5496 - accuracy: 0.4523
457/1563 [=======>......................] - ETA: 1s - loss: 1.5504 - accuracy: 0.4519
495/1563 [========>.....................] - ETA: 1s - loss: 1.5538 - accuracy: 0.4509
533/1563 [=========>....................] - ETA: 1s - loss: 1.5513 - accuracy: 0.4523
571/1563 [=========>....................] - ETA: 1s - loss: 1.5482 - accuracy: 0.4538
609/1563 [==========>...................] - ETA: 1s - loss: 1.5487 - accuracy: 0.4516
647/1563 [===========>..................] - ETA: 1s - loss: 1.5497 - accuracy: 0.4519
684/1563 [============>.................] - ETA: 1s - loss: 1.5487 - accuracy: 0.4515
721/1563 [============>.................] - ETA: 1s - loss: 1.5455 - accuracy: 0.4528
758/1563 [=============>................] - ETA: 1s - loss: 1.5441 - accuracy: 0.4527
796/1563 [==============>...............] - ETA: 1s - loss: 1.5451 - accuracy: 0.4516
834/1563 [===============>..............] - ETA: 0s - loss: 1.5450 - accuracy: 0.4525
872/1563 [===============>..............] - ETA: 0s - loss: 1.5456 - accuracy: 0.4528
910/1563 [================>.............] - ETA: 0s - loss: 1.5461 - accuracy: 0.4528
948/1563 [=================>............] - ETA: 0s - loss: 1.5461 - accuracy: 0.4538
986/1563 [=================>............] - ETA: 0s - loss: 1.5458 - accuracy: 0.4549
1024/1563 [==================>...........] - ETA: 0s - loss: 1.5444 - accuracy: 0.4546
1062/1563 [===================>..........] - ETA: 0s - loss: 1.5445 - accuracy: 0.4549
1100/1563 [====================>.........] - ETA: 0s - loss: 1.5450 - accuracy: 0.4549
1138/1563 [====================>.........] - ETA: 0s - loss: 1.5436 - accuracy: 0.4547
1176/1563 [=====================>........] - ETA: 0s - loss: 1.5437 - accuracy: 0.4549
1214/1563 [======================>.......] - ETA: 0s - loss: 1.5423 - accuracy: 0.4553
1252/1563 [=======================>......] - ETA: 0s - loss: 1.5417 - accuracy: 0.4557
1290/1563 [=======================>......] - ETA: 0s - loss: 1.5423 - accuracy: 0.4552
1328/1563 [========================>.....] - ETA: 0s - loss: 1.5411 - accuracy: 0.4556
1366/1563 [=========================>....] - ETA: 0s - loss: 1.5419 - accuracy: 0.4555
1404/1563 [=========================>....] - ETA: 0s - loss: 1.5425 - accuracy: 0.4553
1442/1563 [==========================>...] - ETA: 0s - loss: 1.5425 - accuracy: 0.4556
1480/1563 [===========================>..] - ETA: 0s - loss: 1.5430 - accuracy: 0.4550
1518/1563 [============================>.] - ETA: 0s - loss: 1.5424 - accuracy: 0.4552
1556/1563 [============================>.] - ETA: 0s - loss: 1.5428 - accuracy: 0.4553
1563/1563 [==============================] - 2s 1ms/step - loss: 1.5425 - accuracy: 0.4554
Epoch 4/10
1/1563 [..............................] - ETA: 2s - loss: 1.2497 - accuracy: 0.5625
39/1563 [..............................] - ETA: 2s - loss: 1.5134 - accuracy: 0.4671
76/1563 [>.............................] - ETA: 2s - loss: 1.5006 - accuracy: 0.4733
113/1563 [=>............................] - ETA: 1s - loss: 1.5140 - accuracy: 0.4646
151/1563 [=>............................] - ETA: 1s - loss: 1.5121 - accuracy: 0.4690
189/1563 [==>...........................] - ETA: 1s - loss: 1.5108 - accuracy: 0.4684
227/1563 [===>..........................] - ETA: 1s - loss: 1.5023 - accuracy: 0.4672
265/1563 [====>.........................] - ETA: 1s - loss: 1.4999 - accuracy: 0.4656
303/1563 [====>.........................] - ETA: 1s - loss: 1.4948 - accuracy: 0.4684
341/1563 [=====>........................] - ETA: 1s - loss: 1.4962 - accuracy: 0.4668
379/1563 [======>.......................] - ETA: 1s - loss: 1.4989 - accuracy: 0.4672
417/1563 [=======>......................] - ETA: 1s - loss: 1.4951 - accuracy: 0.4695
455/1563 [=======>......................] - ETA: 1s - loss: 1.4924 - accuracy: 0.4701
493/1563 [========>.....................] - ETA: 1s - loss: 1.4898 - accuracy: 0.4714
531/1563 [=========>....................] - ETA: 1s - loss: 1.4922 - accuracy: 0.4715
569/1563 [=========>....................] - ETA: 1s - loss: 1.4904 - accuracy: 0.4719
606/1563 [==========>...................] - ETA: 1s - loss: 1.4911 - accuracy: 0.4716
643/1563 [===========>..................] - ETA: 1s - loss: 1.4914 - accuracy: 0.4719
680/1563 [============>.................] - ETA: 1s - loss: 1.4891 - accuracy: 0.4730
717/1563 [============>.................] - ETA: 1s - loss: 1.4888 - accuracy: 0.4725
755/1563 [=============>................] - ETA: 1s - loss: 1.4892 - accuracy: 0.4721
793/1563 [==============>...............] - ETA: 1s - loss: 1.4898 - accuracy: 0.4720
831/1563 [==============>...............] - ETA: 0s - loss: 1.4894 - accuracy: 0.4722
869/1563 [===============>..............] - ETA: 0s - loss: 1.4876 - accuracy: 0.4724
906/1563 [================>.............] - ETA: 0s - loss: 1.4881 - accuracy: 0.4722
944/1563 [=================>............] - ETA: 0s - loss: 1.4872 - accuracy: 0.4730
982/1563 [=================>............] - ETA: 0s - loss: 1.4882 - accuracy: 0.4733
1020/1563 [==================>...........] - ETA: 0s - loss: 1.4872 - accuracy: 0.4737
1058/1563 [===================>..........] - ETA: 0s - loss: 1.4870 - accuracy: 0.4738
1095/1563 [====================>.........] - ETA: 0s - loss: 1.4874 - accuracy: 0.4742
1133/1563 [====================>.........] - ETA: 0s - loss: 1.4875 - accuracy: 0.4745
1171/1563 [=====================>........] - ETA: 0s - loss: 1.4872 - accuracy: 0.4747
1208/1563 [======================>.......] - ETA: 0s - loss: 1.4867 - accuracy: 0.4744
1245/1563 [======================>.......] - ETA: 0s - loss: 1.4865 - accuracy: 0.4744
1283/1563 [=======================>......] - ETA: 0s - loss: 1.4867 - accuracy: 0.4745
1321/1563 [========================>.....] - ETA: 0s - loss: 1.4877 - accuracy: 0.4747
1359/1563 [=========================>....] - ETA: 0s - loss: 1.4877 - accuracy: 0.4751
1397/1563 [=========================>....] - ETA: 0s - loss: 1.4867 - accuracy: 0.4756
1435/1563 [==========================>...] - ETA: 0s - loss: 1.4862 - accuracy: 0.4752
1473/1563 [===========================>..] - ETA: 0s - loss: 1.4860 - accuracy: 0.4749
1511/1563 [============================>.] - ETA: 0s - loss: 1.4836 - accuracy: 0.4762
1549/1563 [============================>.] - ETA: 0s - loss: 1.4821 - accuracy: 0.4766
1563/1563 [==============================] - 2s 1ms/step - loss: 1.4819 - accuracy: 0.4767
Epoch 5/10
1/1563 [..............................] - ETA: 2s - loss: 1.4868 - accuracy: 0.4062
39/1563 [..............................] - ETA: 2s - loss: 1.4609 - accuracy: 0.4888
77/1563 [>.............................] - ETA: 1s - loss: 1.4837 - accuracy: 0.4842
115/1563 [=>............................] - ETA: 1s - loss: 1.4713 - accuracy: 0.4864
153/1563 [=>............................] - ETA: 1s - loss: 1.4679 - accuracy: 0.4859
191/1563 [==>...........................] - ETA: 1s - loss: 1.4619 - accuracy: 0.4864
228/1563 [===>..........................] - ETA: 1s - loss: 1.4610 - accuracy: 0.4853
265/1563 [====>.........................] - ETA: 1s - loss: 1.4566 - accuracy: 0.4868
303/1563 [====>.........................] - ETA: 1s - loss: 1.4546 - accuracy: 0.4874
340/1563 [=====>........................] - ETA: 1s - loss: 1.4546 - accuracy: 0.4875
378/1563 [======>.......................] - ETA: 1s - loss: 1.4514 - accuracy: 0.4892
416/1563 [======>.......................] - ETA: 1s - loss: 1.4458 - accuracy: 0.4905
453/1563 [=======>......................] - ETA: 1s - loss: 1.4422 - accuracy: 0.4913
491/1563 [========>.....................] - ETA: 1s - loss: 1.4417 - accuracy: 0.4914
529/1563 [=========>....................] - ETA: 1s - loss: 1.4392 - accuracy: 0.4927
564/1563 [=========>....................] - ETA: 1s - loss: 1.4397 - accuracy: 0.4930
602/1563 [==========>...................] - ETA: 1s - loss: 1.4422 - accuracy: 0.4929
639/1563 [===========>..................] - ETA: 1s - loss: 1.4424 - accuracy: 0.4926
677/1563 [===========>..................] - ETA: 1s - loss: 1.4421 - accuracy: 0.4933
715/1563 [============>.................] - ETA: 1s - loss: 1.4424 - accuracy: 0.4930
753/1563 [=============>................] - ETA: 1s - loss: 1.4418 - accuracy: 0.4929
791/1563 [==============>...............] - ETA: 1s - loss: 1.4374 - accuracy: 0.4950
829/1563 [==============>...............] - ETA: 0s - loss: 1.4372 - accuracy: 0.4940
867/1563 [===============>..............] - ETA: 0s - loss: 1.4360 - accuracy: 0.4944
905/1563 [================>.............] - ETA: 0s - loss: 1.4366 - accuracy: 0.4930
942/1563 [=================>............] - ETA: 0s - loss: 1.4370 - accuracy: 0.4927
980/1563 [=================>............] - ETA: 0s - loss: 1.4354 - accuracy: 0.4933
1018/1563 [==================>...........] - ETA: 0s - loss: 1.4347 - accuracy: 0.4934
1055/1563 [===================>..........] - ETA: 0s - loss: 1.4348 - accuracy: 0.4936
1093/1563 [===================>..........] - ETA: 0s - loss: 1.4337 - accuracy: 0.4941
1131/1563 [====================>.........] - ETA: 0s - loss: 1.4333 - accuracy: 0.4943
1169/1563 [=====================>........] - ETA: 0s - loss: 1.4332 - accuracy: 0.4945
1207/1563 [======================>.......] - ETA: 0s - loss: 1.4341 - accuracy: 0.4940
1245/1563 [======================>.......] - ETA: 0s - loss: 1.4332 - accuracy: 0.4945
1283/1563 [=======================>......] - ETA: 0s - loss: 1.4339 - accuracy: 0.4944
1321/1563 [========================>.....] - ETA: 0s - loss: 1.4330 - accuracy: 0.4946
1359/1563 [=========================>....] - ETA: 0s - loss: 1.4334 - accuracy: 0.4948
1397/1563 [=========================>....] - ETA: 0s - loss: 1.4339 - accuracy: 0.4943
1435/1563 [==========================>...] - ETA: 0s - loss: 1.4326 - accuracy: 0.4949
1473/1563 [===========================>..] - ETA: 0s - loss: 1.4335 - accuracy: 0.4945
1511/1563 [============================>.] - ETA: 0s - loss: 1.4333 - accuracy: 0.4948
1549/1563 [============================>.] - ETA: 0s - loss: 1.4336 - accuracy: 0.4944
1563/1563 [==============================] - 2s 1ms/step - loss: 1.4336 - accuracy: 0.4944
Epoch 6/10
1/1563 [..............................] - ETA: 2s - loss: 1.4322 - accuracy: 0.4375
39/1563 [..............................] - ETA: 2s - loss: 1.4632 - accuracy: 0.4824
76/1563 [>.............................] - ETA: 2s - loss: 1.4226 - accuracy: 0.4975
114/1563 [=>............................] - ETA: 1s - loss: 1.4029 - accuracy: 0.5055
152/1563 [=>............................] - ETA: 1s - loss: 1.3849 - accuracy: 0.5082
190/1563 [==>...........................] - ETA: 1s - loss: 1.3966 - accuracy: 0.5043
228/1563 [===>..........................] - ETA: 1s - loss: 1.3954 - accuracy: 0.5096
266/1563 [====>.........................] - ETA: 1s - loss: 1.3959 - accuracy: 0.5083
303/1563 [====>.........................] - ETA: 1s - loss: 1.3932 - accuracy: 0.5100
341/1563 [=====>........................] - ETA: 1s - loss: 1.4017 - accuracy: 0.5059
379/1563 [======>.......................] - ETA: 1s - loss: 1.3969 - accuracy: 0.5091
417/1563 [=======>......................] - ETA: 1s - loss: 1.3968 - accuracy: 0.5107
455/1563 [=======>......................] - ETA: 1s - loss: 1.3994 - accuracy: 0.5106
493/1563 [========>.....................] - ETA: 1s - loss: 1.3955 - accuracy: 0.5108
531/1563 [=========>....................] - ETA: 1s - loss: 1.3966 - accuracy: 0.5102
569/1563 [=========>....................] - ETA: 1s - loss: 1.3987 - accuracy: 0.5103
607/1563 [==========>...................] - ETA: 1s - loss: 1.3999 - accuracy: 0.5096
645/1563 [===========>..................] - ETA: 1s - loss: 1.3971 - accuracy: 0.5101
683/1563 [============>.................] - ETA: 1s - loss: 1.3969 - accuracy: 0.5099
720/1563 [============>.................] - ETA: 1s - loss: 1.3955 - accuracy: 0.5101
758/1563 [=============>................] - ETA: 1s - loss: 1.3948 - accuracy: 0.5099
796/1563 [==============>...............] - ETA: 1s - loss: 1.3950 - accuracy: 0.5099
834/1563 [===============>..............] - ETA: 0s - loss: 1.3942 - accuracy: 0.5102
872/1563 [===============>..............] - ETA: 0s - loss: 1.3932 - accuracy: 0.5113
909/1563 [================>.............] - ETA: 0s - loss: 1.3937 - accuracy: 0.5113
947/1563 [=================>............] - ETA: 0s - loss: 1.3932 - accuracy: 0.5111
985/1563 [=================>............] - ETA: 0s - loss: 1.3936 - accuracy: 0.5111
1023/1563 [==================>...........] - ETA: 0s - loss: 1.3914 - accuracy: 0.5117
1061/1563 [===================>..........] - ETA: 0s - loss: 1.3909 - accuracy: 0.5118
1099/1563 [====================>.........] - ETA: 0s - loss: 1.3913 - accuracy: 0.5118
1137/1563 [====================>.........] - ETA: 0s - loss: 1.3903 - accuracy: 0.5122
1174/1563 [=====================>........] - ETA: 0s - loss: 1.3886 - accuracy: 0.5126
1211/1563 [======================>.......] - ETA: 0s - loss: 1.3893 - accuracy: 0.5125
1248/1563 [======================>.......] - ETA: 0s - loss: 1.3885 - accuracy: 0.5128
1286/1563 [=======================>......] - ETA: 0s - loss: 1.3886 - accuracy: 0.5128
1324/1563 [========================>.....] - ETA: 0s - loss: 1.3890 - accuracy: 0.5125
1362/1563 [=========================>....] - ETA: 0s - loss: 1.3895 - accuracy: 0.5126
1400/1563 [=========================>....] - ETA: 0s - loss: 1.3891 - accuracy: 0.5127
1438/1563 [==========================>...] - ETA: 0s - loss: 1.3881 - accuracy: 0.5132
1473/1563 [===========================>..] - ETA: 0s - loss: 1.3888 - accuracy: 0.5128
1510/1563 [===========================>..] - ETA: 0s - loss: 1.3881 - accuracy: 0.5128
1548/1563 [============================>.] - ETA: 0s - loss: 1.3888 - accuracy: 0.5127
1563/1563 [==============================] - 2s 1ms/step - loss: 1.3890 - accuracy: 0.5126
Epoch 7/10
1/1563 [..............................] - ETA: 2s - loss: 1.3252 - accuracy: 0.5625
38/1563 [..............................] - ETA: 2s - loss: 1.3660 - accuracy: 0.5058
75/1563 [>.............................] - ETA: 2s - loss: 1.3466 - accuracy: 0.5258
113/1563 [=>............................] - ETA: 1s - loss: 1.3714 - accuracy: 0.5196
151/1563 [=>............................] - ETA: 1s - loss: 1.3673 - accuracy: 0.5186
189/1563 [==>...........................] - ETA: 1s - loss: 1.3589 - accuracy: 0.5203
227/1563 [===>..........................] - ETA: 1s - loss: 1.3605 - accuracy: 0.5209
265/1563 [====>.........................] - ETA: 1s - loss: 1.3528 - accuracy: 0.5230
303/1563 [====>.........................] - ETA: 1s - loss: 1.3468 - accuracy: 0.5239
341/1563 [=====>........................] - ETA: 1s - loss: 1.3479 - accuracy: 0.5247
379/1563 [======>.......................] - ETA: 1s - loss: 1.3483 - accuracy: 0.5242
417/1563 [=======>......................] - ETA: 1s - loss: 1.3447 - accuracy: 0.5241
455/1563 [=======>......................] - ETA: 1s - loss: 1.3430 - accuracy: 0.5240
493/1563 [========>.....................] - ETA: 1s - loss: 1.3441 - accuracy: 0.5245
531/1563 [=========>....................] - ETA: 1s - loss: 1.3454 - accuracy: 0.5232
569/1563 [=========>....................] - ETA: 1s - loss: 1.3456 - accuracy: 0.5239
607/1563 [==========>...................] - ETA: 1s - loss: 1.3438 - accuracy: 0.5253
645/1563 [===========>..................] - ETA: 1s - loss: 1.3434 - accuracy: 0.5262
683/1563 [============>.................] - ETA: 1s - loss: 1.3463 - accuracy: 0.5256
721/1563 [============>.................] - ETA: 1s - loss: 1.3472 - accuracy: 0.5251
759/1563 [=============>................] - ETA: 1s - loss: 1.3451 - accuracy: 0.5263
797/1563 [==============>...............] - ETA: 1s - loss: 1.3451 - accuracy: 0.5264
835/1563 [===============>..............] - ETA: 0s - loss: 1.3438 - accuracy: 0.5268
872/1563 [===============>..............] - ETA: 0s - loss: 1.3445 - accuracy: 0.5262
910/1563 [================>.............] - ETA: 0s - loss: 1.3445 - accuracy: 0.5260
947/1563 [=================>............] - ETA: 0s - loss: 1.3445 - accuracy: 0.5267
985/1563 [=================>............] - ETA: 0s - loss: 1.3441 - accuracy: 0.5267
1023/1563 [==================>...........] - ETA: 0s - loss: 1.3459 - accuracy: 0.5258
1061/1563 [===================>..........] - ETA: 0s - loss: 1.3454 - accuracy: 0.5262
1099/1563 [====================>.........] - ETA: 0s - loss: 1.3467 - accuracy: 0.5265
1137/1563 [====================>.........] - ETA: 0s - loss: 1.3472 - accuracy: 0.5270
1175/1563 [=====================>........] - ETA: 0s - loss: 1.3475 - accuracy: 0.5270
1213/1563 [======================>.......] - ETA: 0s - loss: 1.3479 - accuracy: 0.5269
1251/1563 [=======================>......] - ETA: 0s - loss: 1.3485 - accuracy: 0.5263
1288/1563 [=======================>......] - ETA: 0s - loss: 1.3481 - accuracy: 0.5268
1326/1563 [========================>.....] - ETA: 0s - loss: 1.3481 - accuracy: 0.5262
1364/1563 [=========================>....] - ETA: 0s - loss: 1.3492 - accuracy: 0.5258
1402/1563 [=========================>....] - ETA: 0s - loss: 1.3498 - accuracy: 0.5253
1440/1563 [==========================>...] - ETA: 0s - loss: 1.3513 - accuracy: 0.5249
1476/1563 [===========================>..] - ETA: 0s - loss: 1.3507 - accuracy: 0.5251
1514/1563 [============================>.] - ETA: 0s - loss: 1.3493 - accuracy: 0.5256
1552/1563 [============================>.] - ETA: 0s - loss: 1.3486 - accuracy: 0.5260
1563/1563 [==============================] - 2s 1ms/step - loss: 1.3485 - accuracy: 0.5262
Epoch 8/10
1/1563 [..............................] - ETA: 2s - loss: 1.2186 - accuracy: 0.5312
39/1563 [..............................] - ETA: 2s - loss: 1.3463 - accuracy: 0.5321
77/1563 [>.............................] - ETA: 1s - loss: 1.3316 - accuracy: 0.5386
115/1563 [=>............................] - ETA: 1s - loss: 1.3119 - accuracy: 0.5459
153/1563 [=>............................] - ETA: 1s - loss: 1.3146 - accuracy: 0.5419
191/1563 [==>...........................] - ETA: 1s - loss: 1.3181 - accuracy: 0.5416
229/1563 [===>..........................] - ETA: 1s - loss: 1.3137 - accuracy: 0.5403
267/1563 [====>.........................] - ETA: 1s - loss: 1.3143 - accuracy: 0.5383
305/1563 [====>.........................] - ETA: 1s - loss: 1.3189 - accuracy: 0.5376
343/1563 [=====>........................] - ETA: 1s - loss: 1.3186 - accuracy: 0.5385
381/1563 [======>.......................] - ETA: 1s - loss: 1.3162 - accuracy: 0.5372
419/1563 [=======>......................] - ETA: 1s - loss: 1.3144 - accuracy: 0.5374
457/1563 [=======>......................] - ETA: 1s - loss: 1.3158 - accuracy: 0.5370
494/1563 [========>.....................] - ETA: 1s - loss: 1.3207 - accuracy: 0.5345
531/1563 [=========>....................] - ETA: 1s - loss: 1.3190 - accuracy: 0.5343
569/1563 [=========>....................] - ETA: 1s - loss: 1.3201 - accuracy: 0.5347
606/1563 [==========>...................] - ETA: 1s - loss: 1.3168 - accuracy: 0.5363
644/1563 [===========>..................] - ETA: 1s - loss: 1.3193 - accuracy: 0.5351
682/1563 [============>.................] - ETA: 1s - loss: 1.3197 - accuracy: 0.5345
720/1563 [============>.................] - ETA: 1s - loss: 1.3219 - accuracy: 0.5334
758/1563 [=============>................] - ETA: 1s - loss: 1.3203 - accuracy: 0.5338
796/1563 [==============>...............] - ETA: 1s - loss: 1.3224 - accuracy: 0.5338
834/1563 [===============>..............] - ETA: 0s - loss: 1.3223 - accuracy: 0.5341
872/1563 [===============>..............] - ETA: 0s - loss: 1.3221 - accuracy: 0.5338
910/1563 [================>.............] - ETA: 0s - loss: 1.3190 - accuracy: 0.5353
948/1563 [=================>............] - ETA: 0s - loss: 1.3170 - accuracy: 0.5354
986/1563 [=================>............] - ETA: 0s - loss: 1.3173 - accuracy: 0.5351
1024/1563 [==================>...........] - ETA: 0s - loss: 1.3166 - accuracy: 0.5356
1062/1563 [===================>..........] - ETA: 0s - loss: 1.3157 - accuracy: 0.5365
1100/1563 [====================>.........] - ETA: 0s - loss: 1.3152 - accuracy: 0.5370
1138/1563 [====================>.........] - ETA: 0s - loss: 1.3157 - accuracy: 0.5369
1176/1563 [=====================>........] - ETA: 0s - loss: 1.3151 - accuracy: 0.5377
1214/1563 [======================>.......] - ETA: 0s - loss: 1.3148 - accuracy: 0.5385
1252/1563 [=======================>......] - ETA: 0s - loss: 1.3157 - accuracy: 0.5378
1290/1563 [=======================>......] - ETA: 0s - loss: 1.3132 - accuracy: 0.5387
1328/1563 [========================>.....] - ETA: 0s - loss: 1.3132 - accuracy: 0.5384
1366/1563 [=========================>....] - ETA: 0s - loss: 1.3125 - accuracy: 0.5382
1404/1563 [=========================>....] - ETA: 0s - loss: 1.3148 - accuracy: 0.5378
1442/1563 [==========================>...] - ETA: 0s - loss: 1.3148 - accuracy: 0.5375
1480/1563 [===========================>..] - ETA: 0s - loss: 1.3140 - accuracy: 0.5378
1518/1563 [============================>.] - ETA: 0s - loss: 1.3151 - accuracy: 0.5376
1556/1563 [============================>.] - ETA: 0s - loss: 1.3164 - accuracy: 0.5373
1563/1563 [==============================] - 2s 1ms/step - loss: 1.3165 - accuracy: 0.5373
Epoch 9/10
1/1563 [..............................] - ETA: 2s - loss: 1.5293 - accuracy: 0.4375
39/1563 [..............................] - ETA: 2s - loss: 1.2910 - accuracy: 0.5449
77/1563 [>.............................] - ETA: 1s - loss: 1.2972 - accuracy: 0.5463
115/1563 [=>............................] - ETA: 1s - loss: 1.3142 - accuracy: 0.5405
153/1563 [=>............................] - ETA: 1s - loss: 1.3160 - accuracy: 0.5404
191/1563 [==>...........................] - ETA: 1s - loss: 1.3056 - accuracy: 0.5456
228/1563 [===>..........................] - ETA: 1s - loss: 1.3002 - accuracy: 0.5461
265/1563 [====>.........................] - ETA: 1s - loss: 1.2986 - accuracy: 0.5447
302/1563 [====>.........................] - ETA: 1s - loss: 1.2985 - accuracy: 0.5446
340/1563 [=====>........................] - ETA: 1s - loss: 1.3025 - accuracy: 0.5426
378/1563 [======>.......................] - ETA: 1s - loss: 1.3002 - accuracy: 0.5449
416/1563 [======>.......................] - ETA: 1s - loss: 1.2959 - accuracy: 0.5460
453/1563 [=======>......................] - ETA: 1s - loss: 1.2923 - accuracy: 0.5462
491/1563 [========>.....................] - ETA: 1s - loss: 1.2919 - accuracy: 0.5469
529/1563 [=========>....................] - ETA: 1s - loss: 1.2869 - accuracy: 0.5486
567/1563 [=========>....................] - ETA: 1s - loss: 1.2898 - accuracy: 0.5476
605/1563 [==========>...................] - ETA: 1s - loss: 1.2888 - accuracy: 0.5478
643/1563 [===========>..................] - ETA: 1s - loss: 1.2862 - accuracy: 0.5481
681/1563 [============>.................] - ETA: 1s - loss: 1.2840 - accuracy: 0.5489
719/1563 [============>.................] - ETA: 1s - loss: 1.2846 - accuracy: 0.5489
757/1563 [=============>................] - ETA: 1s - loss: 1.2858 - accuracy: 0.5492
795/1563 [==============>...............] - ETA: 1s - loss: 1.2828 - accuracy: 0.5506
833/1563 [==============>...............] - ETA: 0s - loss: 1.2826 - accuracy: 0.5512
871/1563 [===============>..............] - ETA: 0s - loss: 1.2823 - accuracy: 0.5510
909/1563 [================>.............] - ETA: 0s - loss: 1.2802 - accuracy: 0.5514
947/1563 [=================>............] - ETA: 0s - loss: 1.2808 - accuracy: 0.5512
985/1563 [=================>............] - ETA: 0s - loss: 1.2802 - accuracy: 0.5515
1023/1563 [==================>...........] - ETA: 0s - loss: 1.2798 - accuracy: 0.5523
1061/1563 [===================>..........] - ETA: 0s - loss: 1.2822 - accuracy: 0.5510
1099/1563 [====================>.........] - ETA: 0s - loss: 1.2818 - accuracy: 0.5512
1137/1563 [====================>.........] - ETA: 0s - loss: 1.2818 - accuracy: 0.5508
1175/1563 [=====================>........] - ETA: 0s - loss: 1.2820 - accuracy: 0.5506
1213/1563 [======================>.......] - ETA: 0s - loss: 1.2847 - accuracy: 0.5499
1251/1563 [=======================>......] - ETA: 0s - loss: 1.2857 - accuracy: 0.5496
1289/1563 [=======================>......] - ETA: 0s - loss: 1.2843 - accuracy: 0.5500
1327/1563 [========================>.....] - ETA: 0s - loss: 1.2837 - accuracy: 0.5502
1365/1563 [=========================>....] - ETA: 0s - loss: 1.2839 - accuracy: 0.5495
1403/1563 [=========================>....] - ETA: 0s - loss: 1.2832 - accuracy: 0.5496
1441/1563 [==========================>...] - ETA: 0s - loss: 1.2832 - accuracy: 0.5498
1478/1563 [===========================>..] - ETA: 0s - loss: 1.2835 - accuracy: 0.5496
1516/1563 [============================>.] - ETA: 0s - loss: 1.2848 - accuracy: 0.5493
1554/1563 [============================>.] - ETA: 0s - loss: 1.2852 - accuracy: 0.5492
1563/1563 [==============================] - 2s 1ms/step - loss: 1.2853 - accuracy: 0.5492
Epoch 10/10
1/1563 [..............................] - ETA: 2s - loss: 1.1180 - accuracy: 0.5625
39/1563 [..............................] - ETA: 2s - loss: 1.2569 - accuracy: 0.5385
77/1563 [>.............................] - ETA: 1s - loss: 1.2752 - accuracy: 0.5394
114/1563 [=>............................] - ETA: 1s - loss: 1.2778 - accuracy: 0.5430
152/1563 [=>............................] - ETA: 1s - loss: 1.2720 - accuracy: 0.5448
190/1563 [==>...........................] - ETA: 1s - loss: 1.2722 - accuracy: 0.5457
228/1563 [===>..........................] - ETA: 1s - loss: 1.2685 - accuracy: 0.5503
266/1563 [====>.........................] - ETA: 1s - loss: 1.2599 - accuracy: 0.5556
303/1563 [====>.........................] - ETA: 1s - loss: 1.2588 - accuracy: 0.5571
340/1563 [=====>........................] - ETA: 1s - loss: 1.2572 - accuracy: 0.5582
378/1563 [======>.......................] - ETA: 1s - loss: 1.2540 - accuracy: 0.5583
416/1563 [======>.......................] - ETA: 1s - loss: 1.2532 - accuracy: 0.5591
453/1563 [=======>......................] - ETA: 1s - loss: 1.2496 - accuracy: 0.5607
491/1563 [========>.....................] - ETA: 1s - loss: 1.2496 - accuracy: 0.5617
528/1563 [=========>....................] - ETA: 1s - loss: 1.2461 - accuracy: 0.5620
565/1563 [=========>....................] - ETA: 1s - loss: 1.2473 - accuracy: 0.5613
603/1563 [==========>...................] - ETA: 1s - loss: 1.2506 - accuracy: 0.5605
641/1563 [===========>..................] - ETA: 1s - loss: 1.2485 - accuracy: 0.5616
679/1563 [============>.................] - ETA: 1s - loss: 1.2495 - accuracy: 0.5611
717/1563 [============>.................] - ETA: 1s - loss: 1.2507 - accuracy: 0.5608
755/1563 [=============>................] - ETA: 1s - loss: 1.2484 - accuracy: 0.5622
793/1563 [==============>...............] - ETA: 1s - loss: 1.2490 - accuracy: 0.5622
831/1563 [==============>...............] - ETA: 0s - loss: 1.2505 - accuracy: 0.5622
869/1563 [===============>..............] - ETA: 0s - loss: 1.2498 - accuracy: 0.5625
907/1563 [================>.............] - ETA: 0s - loss: 1.2499 - accuracy: 0.5625
945/1563 [=================>............] - ETA: 0s - loss: 1.2514 - accuracy: 0.5618
983/1563 [=================>............] - ETA: 0s - loss: 1.2505 - accuracy: 0.5620
1020/1563 [==================>...........] - ETA: 0s - loss: 1.2492 - accuracy: 0.5625
1057/1563 [===================>..........] - ETA: 0s - loss: 1.2479 - accuracy: 0.5632
1095/1563 [====================>.........] - ETA: 0s - loss: 1.2479 - accuracy: 0.5634
1133/1563 [====================>.........] - ETA: 0s - loss: 1.2488 - accuracy: 0.5635
1171/1563 [=====================>........] - ETA: 0s - loss: 1.2470 - accuracy: 0.5640
1208/1563 [======================>.......] - ETA: 0s - loss: 1.2478 - accuracy: 0.5640
1245/1563 [======================>.......] - ETA: 0s - loss: 1.2489 - accuracy: 0.5635
1283/1563 [=======================>......] - ETA: 0s - loss: 1.2500 - accuracy: 0.5624
1321/1563 [========================>.....] - ETA: 0s - loss: 1.2502 - accuracy: 0.5624
1359/1563 [=========================>....] - ETA: 0s - loss: 1.2513 - accuracy: 0.5623
1397/1563 [=========================>....] - ETA: 0s - loss: 1.2538 - accuracy: 0.5611
1435/1563 [==========================>...] - ETA: 0s - loss: 1.2549 - accuracy: 0.5610
1473/1563 [===========================>..] - ETA: 0s - loss: 1.2545 - accuracy: 0.5612
1511/1563 [============================>.] - ETA: 0s - loss: 1.2551 - accuracy: 0.5611
1549/1563 [============================>.] - ETA: 0s - loss: 1.2546 - accuracy: 0.5614
1563/1563 [==============================] - 2s 1ms/step - loss: 1.2546 - accuracy: 0.5617
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
flatten (Flatten) (None, 3072) 0
dense (Dense) (None, 3000) 9219000
dense_1 (Dense) (None, 1000) 3001000
dense_2 (Dense) (None, 10) 10010
=================================================================
Total params: 12,230,010
Trainable params: 12,230,010
Non-trainable params: 0
_________________________________________________________________
None
wall time seconds (without loading test data set)= 45.8397376537323
hydro03
[SSH] completed
[SSH] exit-status: 0
[workspace] $ /bin/sh -xe /tmp/jenkins15382284926932086226.sh
+ scp 'HYDRO_REMOTE:~svchydrojenkins/python/tensorflow_time.txt' /var/lib/jenkins/jobs/tensorflow/workspace
Recording plot data
Saving plot series data from: /var/lib/jenkins/jobs/tensorflow/workspace/tensorflow_time.txt
Finished: SUCCESS
![[Jenkins]](/static/755cd4af/images/svgs/logo.svg){kind=logo}
