Started by timer
Running as SYSTEM
Building in workspace /var/lib/jenkins/jobs/tensorflow/workspace
[SSH] script:
module load anaconda3_gpu
module load cudnn
cd python
rm -f tensorflow_test.txt
srun --nodes=1 --ntasks-per-node=1 --mem=16g \
--time=00:05:00 \
--partition=a100rome \
--account=bbmb-hydro \
--gpus-per-node=1 \
--gpu-bind=closest \
python3 cifar10gpu.py | tee tensorflow_test.txt
YVALUE=`grep "wall time seconds" tensorflow_test.txt | cut -d'=' -f2`
echo YVALUE=$YVALUE > tensorflow_time.txt
[SSH] executing...
srun: job 95795 queued and waiting for resources
srun: job 95795 has been allocated resources
2024-10-28 07:43:07.826208: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-10-28 07:43:32.191531: E tensorflow/stream_executor/cuda/cuda_blas.cc:2981] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-10-28 07:44:06.757844: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /sw/spack/hydrogpu-2022-06/apps/cudnn/8.4.0.27-11.6-gcc-11.3.0-qbhsfav/lib:/sw/spack/hydrogpu-2022-06/apps/cuda/11.7.0-gcc-11.3.0-3ysno6b/lib64:/sw/spack/hydrogpu-2022-06/apps/openmpi/4.1.4-gcc-11.3.0-i6koipj/lib:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib64:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib
2024-10-28 07:44:06.758803: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /sw/spack/hydrogpu-2022-06/apps/cudnn/8.4.0.27-11.6-gcc-11.3.0-qbhsfav/lib:/sw/spack/hydrogpu-2022-06/apps/cuda/11.7.0-gcc-11.3.0-3ysno6b/lib64:/sw/spack/hydrogpu-2022-06/apps/openmpi/4.1.4-gcc-11.3.0-i6koipj/lib:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib64:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib
2024-10-28 07:44:06.758811: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
2024-10-28 07:45:01.149549: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-10-28 07:45:03.791532: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1616] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 79148 MB memory: -> device: 0, name: NVIDIA A100 80GB PCIe, pci bus id: 0000:21:00.0, compute capability: 8.0
(50000, 32, 32, 3) (10000, 32, 32, 3)
Epoch 1/10
2024-10-28 07:45:11.006664: I tensorflow/stream_executor/cuda/cuda_blas.cc:1614] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
1/1563 [..............................] - ETA: 1:40:05 - loss: 2.3634 - accuracy: 0.0938
37/1563 [..............................] - ETA: 2s - loss: 2.2428 - accuracy: 0.1875
74/1563 [>.............................] - ETA: 2s - loss: 2.1808 - accuracy: 0.2048
112/1563 [=>............................] - ETA: 1s - loss: 2.1297 - accuracy: 0.2252
150/1563 [=>............................] - ETA: 1s - loss: 2.0961 - accuracy: 0.2385
188/1563 [==>...........................] - ETA: 1s - loss: 2.0677 - accuracy: 0.2510
225/1563 [===>..........................] - ETA: 1s - loss: 2.0360 - accuracy: 0.2631
263/1563 [====>.........................] - ETA: 1s - loss: 2.0248 - accuracy: 0.2670
301/1563 [====>.........................] - ETA: 1s - loss: 2.0062 - accuracy: 0.2770
339/1563 [=====>........................] - ETA: 1s - loss: 1.9933 - accuracy: 0.2818
377/1563 [======>.......................] - ETA: 1s - loss: 1.9852 - accuracy: 0.2864
415/1563 [======>.......................] - ETA: 1s - loss: 1.9708 - accuracy: 0.2906
453/1563 [=======>......................] - ETA: 1s - loss: 1.9641 - accuracy: 0.2935
491/1563 [========>.....................] - ETA: 1s - loss: 1.9551 - accuracy: 0.2956
529/1563 [=========>....................] - ETA: 1s - loss: 1.9478 - accuracy: 0.2976
567/1563 [=========>....................] - ETA: 1s - loss: 1.9416 - accuracy: 0.3007
605/1563 [==========>...................] - ETA: 1s - loss: 1.9305 - accuracy: 0.3053
643/1563 [===========>..................] - ETA: 1s - loss: 1.9208 - accuracy: 0.3088
681/1563 [============>.................] - ETA: 1s - loss: 1.9129 - accuracy: 0.3118
719/1563 [============>.................] - ETA: 1s - loss: 1.9058 - accuracy: 0.3151
757/1563 [=============>................] - ETA: 1s - loss: 1.9012 - accuracy: 0.3170
795/1563 [==============>...............] - ETA: 1s - loss: 1.8950 - accuracy: 0.3199
833/1563 [==============>...............] - ETA: 0s - loss: 1.8899 - accuracy: 0.3216
871/1563 [===============>..............] - ETA: 0s - loss: 1.8844 - accuracy: 0.3248
909/1563 [================>.............] - ETA: 0s - loss: 1.8782 - accuracy: 0.3277
947/1563 [=================>............] - ETA: 0s - loss: 1.8731 - accuracy: 0.3294
985/1563 [=================>............] - ETA: 0s - loss: 1.8683 - accuracy: 0.3319
1023/1563 [==================>...........] - ETA: 0s - loss: 1.8638 - accuracy: 0.3334
1061/1563 [===================>..........] - ETA: 0s - loss: 1.8583 - accuracy: 0.3355
1099/1563 [====================>.........] - ETA: 0s - loss: 1.8536 - accuracy: 0.3380
1137/1563 [====================>.........] - ETA: 0s - loss: 1.8495 - accuracy: 0.3397
1175/1563 [=====================>........] - ETA: 0s - loss: 1.8459 - accuracy: 0.3409
1213/1563 [======================>.......] - ETA: 0s - loss: 1.8426 - accuracy: 0.3427
1251/1563 [=======================>......] - ETA: 0s - loss: 1.8402 - accuracy: 0.3439
1289/1563 [=======================>......] - ETA: 0s - loss: 1.8362 - accuracy: 0.3459
1327/1563 [========================>.....] - ETA: 0s - loss: 1.8326 - accuracy: 0.3469
1365/1563 [=========================>....] - ETA: 0s - loss: 1.8300 - accuracy: 0.3479
1403/1563 [=========================>....] - ETA: 0s - loss: 1.8272 - accuracy: 0.3490
1441/1563 [==========================>...] - ETA: 0s - loss: 1.8242 - accuracy: 0.3504
1479/1563 [===========================>..] - ETA: 0s - loss: 1.8212 - accuracy: 0.3519
1517/1563 [============================>.] - ETA: 0s - loss: 1.8167 - accuracy: 0.3536
1555/1563 [============================>.] - ETA: 0s - loss: 1.8128 - accuracy: 0.3550
1563/1563 [==============================] - 6s 1ms/step - loss: 1.8124 - accuracy: 0.3551
Epoch 2/10
1/1563 [..............................] - ETA: 2s - loss: 1.9957 - accuracy: 0.3125
39/1563 [..............................] - ETA: 2s - loss: 1.6620 - accuracy: 0.4159
77/1563 [>.............................] - ETA: 1s - loss: 1.6504 - accuracy: 0.4156
115/1563 [=>............................] - ETA: 1s - loss: 1.6642 - accuracy: 0.4168
153/1563 [=>............................] - ETA: 1s - loss: 1.6619 - accuracy: 0.4140
191/1563 [==>...........................] - ETA: 1s - loss: 1.6559 - accuracy: 0.4157
229/1563 [===>..........................] - ETA: 1s - loss: 1.6586 - accuracy: 0.4139
267/1563 [====>.........................] - ETA: 1s - loss: 1.6543 - accuracy: 0.4190
305/1563 [====>.........................] - ETA: 1s - loss: 1.6532 - accuracy: 0.4187
343/1563 [=====>........................] - ETA: 1s - loss: 1.6484 - accuracy: 0.4208
381/1563 [======>.......................] - ETA: 1s - loss: 1.6481 - accuracy: 0.4210
419/1563 [=======>......................] - ETA: 1s - loss: 1.6421 - accuracy: 0.4224
457/1563 [=======>......................] - ETA: 1s - loss: 1.6427 - accuracy: 0.4220
495/1563 [========>.....................] - ETA: 1s - loss: 1.6449 - accuracy: 0.4227
533/1563 [=========>....................] - ETA: 1s - loss: 1.6430 - accuracy: 0.4227
571/1563 [=========>....................] - ETA: 1s - loss: 1.6440 - accuracy: 0.4217
609/1563 [==========>...................] - ETA: 1s - loss: 1.6433 - accuracy: 0.4215
647/1563 [===========>..................] - ETA: 1s - loss: 1.6425 - accuracy: 0.4226
685/1563 [============>.................] - ETA: 1s - loss: 1.6425 - accuracy: 0.4225
723/1563 [============>.................] - ETA: 1s - loss: 1.6417 - accuracy: 0.4222
761/1563 [=============>................] - ETA: 1s - loss: 1.6396 - accuracy: 0.4230
799/1563 [==============>...............] - ETA: 1s - loss: 1.6369 - accuracy: 0.4234
837/1563 [===============>..............] - ETA: 0s - loss: 1.6370 - accuracy: 0.4233
875/1563 [===============>..............] - ETA: 0s - loss: 1.6365 - accuracy: 0.4234
913/1563 [================>.............] - ETA: 0s - loss: 1.6371 - accuracy: 0.4231
951/1563 [=================>............] - ETA: 0s - loss: 1.6354 - accuracy: 0.4239
989/1563 [=================>............] - ETA: 0s - loss: 1.6347 - accuracy: 0.4238
1027/1563 [==================>...........] - ETA: 0s - loss: 1.6340 - accuracy: 0.4237
1065/1563 [===================>..........] - ETA: 0s - loss: 1.6342 - accuracy: 0.4235
1103/1563 [====================>.........] - ETA: 0s - loss: 1.6327 - accuracy: 0.4237
1141/1563 [====================>.........] - ETA: 0s - loss: 1.6315 - accuracy: 0.4236
1179/1563 [=====================>........] - ETA: 0s - loss: 1.6301 - accuracy: 0.4244
1217/1563 [======================>.......] - ETA: 0s - loss: 1.6294 - accuracy: 0.4244
1255/1563 [=======================>......] - ETA: 0s - loss: 1.6292 - accuracy: 0.4246
1293/1563 [=======================>......] - ETA: 0s - loss: 1.6288 - accuracy: 0.4247
1331/1563 [========================>.....] - ETA: 0s - loss: 1.6284 - accuracy: 0.4250
1369/1563 [=========================>....] - ETA: 0s - loss: 1.6279 - accuracy: 0.4257
1407/1563 [==========================>...] - ETA: 0s - loss: 1.6273 - accuracy: 0.4259
1445/1563 [==========================>...] - ETA: 0s - loss: 1.6270 - accuracy: 0.4252
1483/1563 [===========================>..] - ETA: 0s - loss: 1.6245 - accuracy: 0.4262
1521/1563 [============================>.] - ETA: 0s - loss: 1.6236 - accuracy: 0.4268
1559/1563 [============================>.] - ETA: 0s - loss: 1.6223 - accuracy: 0.4272
1563/1563 [==============================] - 2s 1ms/step - loss: 1.6219 - accuracy: 0.4273
Epoch 3/10
1/1563 [..............................] - ETA: 2s - loss: 1.6275 - accuracy: 0.5000
39/1563 [..............................] - ETA: 2s - loss: 1.5296 - accuracy: 0.4728
77/1563 [>.............................] - ETA: 1s - loss: 1.5540 - accuracy: 0.4521
115/1563 [=>............................] - ETA: 1s - loss: 1.5523 - accuracy: 0.4484
153/1563 [=>............................] - ETA: 1s - loss: 1.5607 - accuracy: 0.4481
191/1563 [==>...........................] - ETA: 1s - loss: 1.5607 - accuracy: 0.4460
229/1563 [===>..........................] - ETA: 1s - loss: 1.5579 - accuracy: 0.4443
267/1563 [====>.........................] - ETA: 1s - loss: 1.5545 - accuracy: 0.4439
305/1563 [====>.........................] - ETA: 1s - loss: 1.5524 - accuracy: 0.4464
343/1563 [=====>........................] - ETA: 1s - loss: 1.5585 - accuracy: 0.4440
381/1563 [======>.......................] - ETA: 1s - loss: 1.5572 - accuracy: 0.4450
419/1563 [=======>......................] - ETA: 1s - loss: 1.5607 - accuracy: 0.4429
457/1563 [=======>......................] - ETA: 1s - loss: 1.5604 - accuracy: 0.4419
495/1563 [========>.....................] - ETA: 1s - loss: 1.5623 - accuracy: 0.4415
533/1563 [=========>....................] - ETA: 1s - loss: 1.5612 - accuracy: 0.4420
571/1563 [=========>....................] - ETA: 1s - loss: 1.5592 - accuracy: 0.4435
609/1563 [==========>...................] - ETA: 1s - loss: 1.5562 - accuracy: 0.4454
647/1563 [===========>..................] - ETA: 1s - loss: 1.5549 - accuracy: 0.4475
685/1563 [============>.................] - ETA: 1s - loss: 1.5562 - accuracy: 0.4465
723/1563 [============>.................] - ETA: 1s - loss: 1.5555 - accuracy: 0.4473
761/1563 [=============>................] - ETA: 1s - loss: 1.5525 - accuracy: 0.4486
799/1563 [==============>...............] - ETA: 1s - loss: 1.5525 - accuracy: 0.4490
837/1563 [===============>..............] - ETA: 0s - loss: 1.5511 - accuracy: 0.4501
875/1563 [===============>..............] - ETA: 0s - loss: 1.5513 - accuracy: 0.4505
913/1563 [================>.............] - ETA: 0s - loss: 1.5481 - accuracy: 0.4526
951/1563 [=================>............] - ETA: 0s - loss: 1.5480 - accuracy: 0.4530
989/1563 [=================>............] - ETA: 0s - loss: 1.5465 - accuracy: 0.4538
1027/1563 [==================>...........] - ETA: 0s - loss: 1.5475 - accuracy: 0.4536
1065/1563 [===================>..........] - ETA: 0s - loss: 1.5465 - accuracy: 0.4541
1103/1563 [====================>.........] - ETA: 0s - loss: 1.5477 - accuracy: 0.4540
1141/1563 [====================>.........] - ETA: 0s - loss: 1.5460 - accuracy: 0.4540
1179/1563 [=====================>........] - ETA: 0s - loss: 1.5453 - accuracy: 0.4547
1217/1563 [======================>.......] - ETA: 0s - loss: 1.5466 - accuracy: 0.4540
1255/1563 [=======================>......] - ETA: 0s - loss: 1.5454 - accuracy: 0.4540
1293/1563 [=======================>......] - ETA: 0s - loss: 1.5450 - accuracy: 0.4545
1331/1563 [========================>.....] - ETA: 0s - loss: 1.5436 - accuracy: 0.4555
1369/1563 [=========================>....] - ETA: 0s - loss: 1.5434 - accuracy: 0.4558
1407/1563 [==========================>...] - ETA: 0s - loss: 1.5442 - accuracy: 0.4557
1445/1563 [==========================>...] - ETA: 0s - loss: 1.5428 - accuracy: 0.4563
1483/1563 [===========================>..] - ETA: 0s - loss: 1.5432 - accuracy: 0.4562
1521/1563 [============================>.] - ETA: 0s - loss: 1.5425 - accuracy: 0.4567
1559/1563 [============================>.] - ETA: 0s - loss: 1.5415 - accuracy: 0.4571
1563/1563 [==============================] - 2s 1ms/step - loss: 1.5417 - accuracy: 0.4572
Epoch 4/10
1/1563 [..............................] - ETA: 2s - loss: 1.1234 - accuracy: 0.6875
39/1563 [..............................] - ETA: 2s - loss: 1.4751 - accuracy: 0.4832
77/1563 [>.............................] - ETA: 1s - loss: 1.4719 - accuracy: 0.4850
115/1563 [=>............................] - ETA: 1s - loss: 1.4794 - accuracy: 0.4851
153/1563 [=>............................] - ETA: 1s - loss: 1.4807 - accuracy: 0.4816
191/1563 [==>...........................] - ETA: 1s - loss: 1.4819 - accuracy: 0.4833
229/1563 [===>..........................] - ETA: 1s - loss: 1.4748 - accuracy: 0.4844
267/1563 [====>.........................] - ETA: 1s - loss: 1.4735 - accuracy: 0.4843
305/1563 [====>.........................] - ETA: 1s - loss: 1.4788 - accuracy: 0.4823
343/1563 [=====>........................] - ETA: 1s - loss: 1.4776 - accuracy: 0.4842
381/1563 [======>.......................] - ETA: 1s - loss: 1.4769 - accuracy: 0.4853
419/1563 [=======>......................] - ETA: 1s - loss: 1.4784 - accuracy: 0.4849
457/1563 [=======>......................] - ETA: 1s - loss: 1.4818 - accuracy: 0.4843
495/1563 [========>.....................] - ETA: 1s - loss: 1.4803 - accuracy: 0.4847
533/1563 [=========>....................] - ETA: 1s - loss: 1.4843 - accuracy: 0.4823
571/1563 [=========>....................] - ETA: 1s - loss: 1.4845 - accuracy: 0.4813
609/1563 [==========>...................] - ETA: 1s - loss: 1.4847 - accuracy: 0.4805
647/1563 [===========>..................] - ETA: 1s - loss: 1.4892 - accuracy: 0.4790
685/1563 [============>.................] - ETA: 1s - loss: 1.4894 - accuracy: 0.4787
723/1563 [============>.................] - ETA: 1s - loss: 1.4916 - accuracy: 0.4774
761/1563 [=============>................] - ETA: 1s - loss: 1.4875 - accuracy: 0.4793
799/1563 [==============>...............] - ETA: 1s - loss: 1.4845 - accuracy: 0.4798
837/1563 [===============>..............] - ETA: 0s - loss: 1.4837 - accuracy: 0.4795
875/1563 [===============>..............] - ETA: 0s - loss: 1.4825 - accuracy: 0.4805
913/1563 [================>.............] - ETA: 0s - loss: 1.4831 - accuracy: 0.4798
951/1563 [=================>............] - ETA: 0s - loss: 1.4837 - accuracy: 0.4794
989/1563 [=================>............] - ETA: 0s - loss: 1.4831 - accuracy: 0.4788
1027/1563 [==================>...........] - ETA: 0s - loss: 1.4828 - accuracy: 0.4787
1065/1563 [===================>..........] - ETA: 0s - loss: 1.4837 - accuracy: 0.4781
1103/1563 [====================>.........] - ETA: 0s - loss: 1.4832 - accuracy: 0.4787
1141/1563 [====================>.........] - ETA: 0s - loss: 1.4811 - accuracy: 0.4798
1179/1563 [=====================>........] - ETA: 0s - loss: 1.4803 - accuracy: 0.4800
1217/1563 [======================>.......] - ETA: 0s - loss: 1.4806 - accuracy: 0.4796
1255/1563 [=======================>......] - ETA: 0s - loss: 1.4808 - accuracy: 0.4794
1293/1563 [=======================>......] - ETA: 0s - loss: 1.4807 - accuracy: 0.4795
1331/1563 [========================>.....] - ETA: 0s - loss: 1.4810 - accuracy: 0.4798
1369/1563 [=========================>....] - ETA: 0s - loss: 1.4821 - accuracy: 0.4797
1407/1563 [==========================>...] - ETA: 0s - loss: 1.4810 - accuracy: 0.4800
1445/1563 [==========================>...] - ETA: 0s - loss: 1.4809 - accuracy: 0.4805
1483/1563 [===========================>..] - ETA: 0s - loss: 1.4807 - accuracy: 0.4809
1521/1563 [============================>.] - ETA: 0s - loss: 1.4805 - accuracy: 0.4812
1559/1563 [============================>.] - ETA: 0s - loss: 1.4802 - accuracy: 0.4815
1563/1563 [==============================] - 2s 1ms/step - loss: 1.4801 - accuracy: 0.4816
Epoch 5/10
1/1563 [..............................] - ETA: 2s - loss: 1.1359 - accuracy: 0.5938
39/1563 [..............................] - ETA: 2s - loss: 1.4010 - accuracy: 0.5184
77/1563 [>.............................] - ETA: 1s - loss: 1.4132 - accuracy: 0.5093
115/1563 [=>............................] - ETA: 1s - loss: 1.4203 - accuracy: 0.5038
153/1563 [=>............................] - ETA: 1s - loss: 1.4282 - accuracy: 0.5006
191/1563 [==>...........................] - ETA: 1s - loss: 1.4348 - accuracy: 0.4998
229/1563 [===>..........................] - ETA: 1s - loss: 1.4390 - accuracy: 0.4981
267/1563 [====>.........................] - ETA: 1s - loss: 1.4370 - accuracy: 0.4963
305/1563 [====>.........................] - ETA: 1s - loss: 1.4385 - accuracy: 0.4927
343/1563 [=====>........................] - ETA: 1s - loss: 1.4346 - accuracy: 0.4937
381/1563 [======>.......................] - ETA: 1s - loss: 1.4359 - accuracy: 0.4943
419/1563 [=======>......................] - ETA: 1s - loss: 1.4361 - accuracy: 0.4942
457/1563 [=======>......................] - ETA: 1s - loss: 1.4376 - accuracy: 0.4930
495/1563 [========>.....................] - ETA: 1s - loss: 1.4403 - accuracy: 0.4910
533/1563 [=========>....................] - ETA: 1s - loss: 1.4426 - accuracy: 0.4910
571/1563 [=========>....................] - ETA: 1s - loss: 1.4403 - accuracy: 0.4915
609/1563 [==========>...................] - ETA: 1s - loss: 1.4403 - accuracy: 0.4917
647/1563 [===========>..................] - ETA: 1s - loss: 1.4437 - accuracy: 0.4911
685/1563 [============>.................] - ETA: 1s - loss: 1.4436 - accuracy: 0.4896
723/1563 [============>.................] - ETA: 1s - loss: 1.4431 - accuracy: 0.4897
761/1563 [=============>................] - ETA: 1s - loss: 1.4417 - accuracy: 0.4900
799/1563 [==============>...............] - ETA: 1s - loss: 1.4400 - accuracy: 0.4908
837/1563 [===============>..............] - ETA: 0s - loss: 1.4415 - accuracy: 0.4903
875/1563 [===============>..............] - ETA: 0s - loss: 1.4401 - accuracy: 0.4911
913/1563 [================>.............] - ETA: 0s - loss: 1.4388 - accuracy: 0.4920
951/1563 [=================>............] - ETA: 0s - loss: 1.4380 - accuracy: 0.4924
989/1563 [=================>............] - ETA: 0s - loss: 1.4367 - accuracy: 0.4932
1026/1563 [==================>...........] - ETA: 0s - loss: 1.4360 - accuracy: 0.4940
1064/1563 [===================>..........] - ETA: 0s - loss: 1.4348 - accuracy: 0.4946
1102/1563 [====================>.........] - ETA: 0s - loss: 1.4343 - accuracy: 0.4947
1140/1563 [====================>.........] - ETA: 0s - loss: 1.4346 - accuracy: 0.4948
1178/1563 [=====================>........] - ETA: 0s - loss: 1.4332 - accuracy: 0.4948
1216/1563 [======================>.......] - ETA: 0s - loss: 1.4319 - accuracy: 0.4956
1253/1563 [=======================>......] - ETA: 0s - loss: 1.4314 - accuracy: 0.4960
1291/1563 [=======================>......] - ETA: 0s - loss: 1.4305 - accuracy: 0.4964
1329/1563 [========================>.....] - ETA: 0s - loss: 1.4306 - accuracy: 0.4966
1367/1563 [=========================>....] - ETA: 0s - loss: 1.4312 - accuracy: 0.4960
1405/1563 [=========================>....] - ETA: 0s - loss: 1.4309 - accuracy: 0.4962
1443/1563 [==========================>...] - ETA: 0s - loss: 1.4308 - accuracy: 0.4962
1481/1563 [===========================>..] - ETA: 0s - loss: 1.4308 - accuracy: 0.4968
1519/1563 [============================>.] - ETA: 0s - loss: 1.4308 - accuracy: 0.4972
1557/1563 [============================>.] - ETA: 0s - loss: 1.4309 - accuracy: 0.4973
1563/1563 [==============================] - 2s 1ms/step - loss: 1.4312 - accuracy: 0.4973
Epoch 6/10
1/1563 [..............................] - ETA: 2s - loss: 1.4908 - accuracy: 0.4062
39/1563 [..............................] - ETA: 2s - loss: 1.3937 - accuracy: 0.5152
76/1563 [>.............................] - ETA: 2s - loss: 1.4214 - accuracy: 0.5012
114/1563 [=>............................] - ETA: 1s - loss: 1.4098 - accuracy: 0.5052
152/1563 [=>............................] - ETA: 1s - loss: 1.4036 - accuracy: 0.5047
190/1563 [==>...........................] - ETA: 1s - loss: 1.3988 - accuracy: 0.5044
228/1563 [===>..........................] - ETA: 1s - loss: 1.3935 - accuracy: 0.5070
266/1563 [====>.........................] - ETA: 1s - loss: 1.3976 - accuracy: 0.5070
304/1563 [====>.........................] - ETA: 1s - loss: 1.3955 - accuracy: 0.5083
342/1563 [=====>........................] - ETA: 1s - loss: 1.3926 - accuracy: 0.5099
380/1563 [======>.......................] - ETA: 1s - loss: 1.3902 - accuracy: 0.5109
418/1563 [=======>......................] - ETA: 1s - loss: 1.3885 - accuracy: 0.5120
456/1563 [=======>......................] - ETA: 1s - loss: 1.3856 - accuracy: 0.5123
494/1563 [========>.....................] - ETA: 1s - loss: 1.3863 - accuracy: 0.5125
532/1563 [=========>....................] - ETA: 1s - loss: 1.3881 - accuracy: 0.5119
570/1563 [=========>....................] - ETA: 1s - loss: 1.3876 - accuracy: 0.5121
608/1563 [==========>...................] - ETA: 1s - loss: 1.3872 - accuracy: 0.5128
646/1563 [===========>..................] - ETA: 1s - loss: 1.3869 - accuracy: 0.5130
684/1563 [============>.................] - ETA: 1s - loss: 1.3894 - accuracy: 0.5118
722/1563 [============>.................] - ETA: 1s - loss: 1.3886 - accuracy: 0.5127
760/1563 [=============>................] - ETA: 1s - loss: 1.3888 - accuracy: 0.5123
798/1563 [==============>...............] - ETA: 1s - loss: 1.3899 - accuracy: 0.5121
836/1563 [===============>..............] - ETA: 0s - loss: 1.3901 - accuracy: 0.5125
874/1563 [===============>..............] - ETA: 0s - loss: 1.3906 - accuracy: 0.5121
912/1563 [================>.............] - ETA: 0s - loss: 1.3908 - accuracy: 0.5119
950/1563 [=================>............] - ETA: 0s - loss: 1.3930 - accuracy: 0.5118
988/1563 [=================>............] - ETA: 0s - loss: 1.3928 - accuracy: 0.5121
1026/1563 [==================>...........] - ETA: 0s - loss: 1.3920 - accuracy: 0.5121
1064/1563 [===================>..........] - ETA: 0s - loss: 1.3921 - accuracy: 0.5121
1102/1563 [====================>.........] - ETA: 0s - loss: 1.3920 - accuracy: 0.5120
1140/1563 [====================>.........] - ETA: 0s - loss: 1.3934 - accuracy: 0.5116
1178/1563 [=====================>........] - ETA: 0s - loss: 1.3933 - accuracy: 0.5118
1216/1563 [======================>.......] - ETA: 0s - loss: 1.3941 - accuracy: 0.5119
1254/1563 [=======================>......] - ETA: 0s - loss: 1.3925 - accuracy: 0.5122
1292/1563 [=======================>......] - ETA: 0s - loss: 1.3923 - accuracy: 0.5119
1330/1563 [========================>.....] - ETA: 0s - loss: 1.3920 - accuracy: 0.5123
1368/1563 [=========================>....] - ETA: 0s - loss: 1.3914 - accuracy: 0.5132
1406/1563 [=========================>....] - ETA: 0s - loss: 1.3924 - accuracy: 0.5132
1444/1563 [==========================>...] - ETA: 0s - loss: 1.3910 - accuracy: 0.5135
1482/1563 [===========================>..] - ETA: 0s - loss: 1.3911 - accuracy: 0.5132
1520/1563 [============================>.] - ETA: 0s - loss: 1.3905 - accuracy: 0.5132
1558/1563 [============================>.] - ETA: 0s - loss: 1.3910 - accuracy: 0.5129
1563/1563 [==============================] - 2s 1ms/step - loss: 1.3909 - accuracy: 0.5129
Epoch 7/10
1/1563 [..............................] - ETA: 2s - loss: 1.4310 - accuracy: 0.5625
38/1563 [..............................] - ETA: 2s - loss: 1.3686 - accuracy: 0.5345
76/1563 [>.............................] - ETA: 2s - loss: 1.3487 - accuracy: 0.5395
114/1563 [=>............................] - ETA: 1s - loss: 1.3576 - accuracy: 0.5304
152/1563 [=>............................] - ETA: 1s - loss: 1.3550 - accuracy: 0.5335
190/1563 [==>...........................] - ETA: 1s - loss: 1.3532 - accuracy: 0.5336
228/1563 [===>..........................] - ETA: 1s - loss: 1.3553 - accuracy: 0.5293
266/1563 [====>.........................] - ETA: 1s - loss: 1.3497 - accuracy: 0.5307
304/1563 [====>.........................] - ETA: 1s - loss: 1.3506 - accuracy: 0.5315
342/1563 [=====>........................] - ETA: 1s - loss: 1.3484 - accuracy: 0.5319
380/1563 [======>.......................] - ETA: 1s - loss: 1.3496 - accuracy: 0.5308
418/1563 [=======>......................] - ETA: 1s - loss: 1.3496 - accuracy: 0.5295
456/1563 [=======>......................] - ETA: 1s - loss: 1.3513 - accuracy: 0.5296
494/1563 [========>.....................] - ETA: 1s - loss: 1.3489 - accuracy: 0.5297
532/1563 [=========>....................] - ETA: 1s - loss: 1.3497 - accuracy: 0.5291
570/1563 [=========>....................] - ETA: 1s - loss: 1.3516 - accuracy: 0.5287
608/1563 [==========>...................] - ETA: 1s - loss: 1.3486 - accuracy: 0.5297
645/1563 [===========>..................] - ETA: 1s - loss: 1.3463 - accuracy: 0.5305
683/1563 [============>.................] - ETA: 1s - loss: 1.3490 - accuracy: 0.5298
721/1563 [============>.................] - ETA: 1s - loss: 1.3513 - accuracy: 0.5281
759/1563 [=============>................] - ETA: 1s - loss: 1.3520 - accuracy: 0.5278
797/1563 [==============>...............] - ETA: 1s - loss: 1.3487 - accuracy: 0.5287
835/1563 [===============>..............] - ETA: 0s - loss: 1.3482 - accuracy: 0.5293
873/1563 [===============>..............] - ETA: 0s - loss: 1.3472 - accuracy: 0.5296
911/1563 [================>.............] - ETA: 0s - loss: 1.3482 - accuracy: 0.5290
948/1563 [=================>............] - ETA: 0s - loss: 1.3498 - accuracy: 0.5288
986/1563 [=================>............] - ETA: 0s - loss: 1.3484 - accuracy: 0.5299
1024/1563 [==================>...........] - ETA: 0s - loss: 1.3484 - accuracy: 0.5297
1062/1563 [===================>..........] - ETA: 0s - loss: 1.3487 - accuracy: 0.5292
1100/1563 [====================>.........] - ETA: 0s - loss: 1.3488 - accuracy: 0.5287
1137/1563 [====================>.........] - ETA: 0s - loss: 1.3490 - accuracy: 0.5286
1175/1563 [=====================>........] - ETA: 0s - loss: 1.3497 - accuracy: 0.5286
1213/1563 [======================>.......] - ETA: 0s - loss: 1.3511 - accuracy: 0.5279
1251/1563 [=======================>......] - ETA: 0s - loss: 1.3511 - accuracy: 0.5274
1289/1563 [=======================>......] - ETA: 0s - loss: 1.3513 - accuracy: 0.5271
1327/1563 [========================>.....] - ETA: 0s - loss: 1.3511 - accuracy: 0.5272
1365/1563 [=========================>....] - ETA: 0s - loss: 1.3495 - accuracy: 0.5280
1403/1563 [=========================>....] - ETA: 0s - loss: 1.3493 - accuracy: 0.5282
1441/1563 [==========================>...] - ETA: 0s - loss: 1.3482 - accuracy: 0.5288
1479/1563 [===========================>..] - ETA: 0s - loss: 1.3479 - accuracy: 0.5287
1517/1563 [============================>.] - ETA: 0s - loss: 1.3485 - accuracy: 0.5285
1555/1563 [============================>.] - ETA: 0s - loss: 1.3497 - accuracy: 0.5277
1563/1563 [==============================] - 2s 1ms/step - loss: 1.3502 - accuracy: 0.5275
Epoch 8/10
1/1563 [..............................] - ETA: 2s - loss: 1.7188 - accuracy: 0.3750
39/1563 [..............................] - ETA: 2s - loss: 1.3162 - accuracy: 0.5361
77/1563 [>.............................] - ETA: 1s - loss: 1.3143 - accuracy: 0.5345
115/1563 [=>............................] - ETA: 1s - loss: 1.3074 - accuracy: 0.5389
153/1563 [=>............................] - ETA: 1s - loss: 1.3109 - accuracy: 0.5384
191/1563 [==>...........................] - ETA: 1s - loss: 1.3120 - accuracy: 0.5376
229/1563 [===>..........................] - ETA: 1s - loss: 1.3144 - accuracy: 0.5374
267/1563 [====>.........................] - ETA: 1s - loss: 1.3162 - accuracy: 0.5378
305/1563 [====>.........................] - ETA: 1s - loss: 1.3115 - accuracy: 0.5410
343/1563 [=====>........................] - ETA: 1s - loss: 1.3102 - accuracy: 0.5405
381/1563 [======>.......................] - ETA: 1s - loss: 1.3115 - accuracy: 0.5392
419/1563 [=======>......................] - ETA: 1s - loss: 1.3116 - accuracy: 0.5392
457/1563 [=======>......................] - ETA: 1s - loss: 1.3110 - accuracy: 0.5397
495/1563 [========>.....................] - ETA: 1s - loss: 1.3096 - accuracy: 0.5403
533/1563 [=========>....................] - ETA: 1s - loss: 1.3090 - accuracy: 0.5410
570/1563 [=========>....................] - ETA: 1s - loss: 1.3095 - accuracy: 0.5403
607/1563 [==========>...................] - ETA: 1s - loss: 1.3077 - accuracy: 0.5420
645/1563 [===========>..................] - ETA: 1s - loss: 1.3090 - accuracy: 0.5416
683/1563 [============>.................] - ETA: 1s - loss: 1.3081 - accuracy: 0.5415
721/1563 [============>.................] - ETA: 1s - loss: 1.3098 - accuracy: 0.5411
759/1563 [=============>................] - ETA: 1s - loss: 1.3103 - accuracy: 0.5403
797/1563 [==============>...............] - ETA: 1s - loss: 1.3089 - accuracy: 0.5414
835/1563 [===============>..............] - ETA: 0s - loss: 1.3105 - accuracy: 0.5402
873/1563 [===============>..............] - ETA: 0s - loss: 1.3110 - accuracy: 0.5395
911/1563 [================>.............] - ETA: 0s - loss: 1.3088 - accuracy: 0.5401
949/1563 [=================>............] - ETA: 0s - loss: 1.3075 - accuracy: 0.5409
987/1563 [=================>............] - ETA: 0s - loss: 1.3084 - accuracy: 0.5404
1025/1563 [==================>...........] - ETA: 0s - loss: 1.3085 - accuracy: 0.5401
1063/1563 [===================>..........] - ETA: 0s - loss: 1.3083 - accuracy: 0.5403
1101/1563 [====================>.........] - ETA: 0s - loss: 1.3105 - accuracy: 0.5397
1139/1563 [====================>.........] - ETA: 0s - loss: 1.3113 - accuracy: 0.5393
1177/1563 [=====================>........] - ETA: 0s - loss: 1.3127 - accuracy: 0.5387
1215/1563 [======================>.......] - ETA: 0s - loss: 1.3116 - accuracy: 0.5389
1252/1563 [=======================>......] - ETA: 0s - loss: 1.3126 - accuracy: 0.5383
1290/1563 [=======================>......] - ETA: 0s - loss: 1.3118 - accuracy: 0.5387
1328/1563 [========================>.....] - ETA: 0s - loss: 1.3119 - accuracy: 0.5388
1366/1563 [=========================>....] - ETA: 0s - loss: 1.3126 - accuracy: 0.5391
1404/1563 [=========================>....] - ETA: 0s - loss: 1.3131 - accuracy: 0.5390
1442/1563 [==========================>...] - ETA: 0s - loss: 1.3140 - accuracy: 0.5388
1479/1563 [===========================>..] - ETA: 0s - loss: 1.3131 - accuracy: 0.5394
1517/1563 [============================>.] - ETA: 0s - loss: 1.3138 - accuracy: 0.5394
1555/1563 [============================>.] - ETA: 0s - loss: 1.3136 - accuracy: 0.5392
1563/1563 [==============================] - 2s 1ms/step - loss: 1.3133 - accuracy: 0.5392
Epoch 9/10
1/1563 [..............................] - ETA: 2s - loss: 1.3414 - accuracy: 0.5312
39/1563 [..............................] - ETA: 2s - loss: 1.2356 - accuracy: 0.5681
77/1563 [>.............................] - ETA: 1s - loss: 1.2601 - accuracy: 0.5536
115/1563 [=>............................] - ETA: 1s - loss: 1.2561 - accuracy: 0.5568
153/1563 [=>............................] - ETA: 1s - loss: 1.2564 - accuracy: 0.5584
191/1563 [==>...........................] - ETA: 1s - loss: 1.2593 - accuracy: 0.5587
229/1563 [===>..........................] - ETA: 1s - loss: 1.2659 - accuracy: 0.5565
267/1563 [====>.........................] - ETA: 1s - loss: 1.2763 - accuracy: 0.5535
305/1563 [====>.........................] - ETA: 1s - loss: 1.2756 - accuracy: 0.5522
343/1563 [=====>........................] - ETA: 1s - loss: 1.2763 - accuracy: 0.5525
381/1563 [======>.......................] - ETA: 1s - loss: 1.2797 - accuracy: 0.5500
419/1563 [=======>......................] - ETA: 1s - loss: 1.2834 - accuracy: 0.5481
457/1563 [=======>......................] - ETA: 1s - loss: 1.2841 - accuracy: 0.5484
495/1563 [========>.....................] - ETA: 1s - loss: 1.2843 - accuracy: 0.5495
533/1563 [=========>....................] - ETA: 1s - loss: 1.2851 - accuracy: 0.5493
571/1563 [=========>....................] - ETA: 1s - loss: 1.2833 - accuracy: 0.5499
609/1563 [==========>...................] - ETA: 1s - loss: 1.2816 - accuracy: 0.5491
647/1563 [===========>..................] - ETA: 1s - loss: 1.2815 - accuracy: 0.5488
685/1563 [============>.................] - ETA: 1s - loss: 1.2833 - accuracy: 0.5480
723/1563 [============>.................] - ETA: 1s - loss: 1.2813 - accuracy: 0.5498
761/1563 [=============>................] - ETA: 1s - loss: 1.2821 - accuracy: 0.5499
799/1563 [==============>...............] - ETA: 1s - loss: 1.2819 - accuracy: 0.5508
837/1563 [===============>..............] - ETA: 0s - loss: 1.2824 - accuracy: 0.5500
875/1563 [===============>..............] - ETA: 0s - loss: 1.2828 - accuracy: 0.5496
912/1563 [================>.............] - ETA: 0s - loss: 1.2810 - accuracy: 0.5508
950/1563 [=================>............] - ETA: 0s - loss: 1.2819 - accuracy: 0.5503
988/1563 [=================>............] - ETA: 0s - loss: 1.2806 - accuracy: 0.5508
1026/1563 [==================>...........] - ETA: 0s - loss: 1.2810 - accuracy: 0.5507
1064/1563 [===================>..........] - ETA: 0s - loss: 1.2804 - accuracy: 0.5508
1101/1563 [====================>.........] - ETA: 0s - loss: 1.2809 - accuracy: 0.5505
1139/1563 [====================>.........] - ETA: 0s - loss: 1.2791 - accuracy: 0.5514
1177/1563 [=====================>........] - ETA: 0s - loss: 1.2782 - accuracy: 0.5515
1214/1563 [======================>.......] - ETA: 0s - loss: 1.2786 - accuracy: 0.5517
1252/1563 [=======================>......] - ETA: 0s - loss: 1.2788 - accuracy: 0.5518
1290/1563 [=======================>......] - ETA: 0s - loss: 1.2793 - accuracy: 0.5512
1328/1563 [========================>.....] - ETA: 0s - loss: 1.2807 - accuracy: 0.5507
1366/1563 [=========================>....] - ETA: 0s - loss: 1.2811 - accuracy: 0.5506
1403/1563 [=========================>....] - ETA: 0s - loss: 1.2811 - accuracy: 0.5504
1441/1563 [==========================>...] - ETA: 0s - loss: 1.2819 - accuracy: 0.5500
1478/1563 [===========================>..] - ETA: 0s - loss: 1.2818 - accuracy: 0.5502
1516/1563 [============================>.] - ETA: 0s - loss: 1.2813 - accuracy: 0.5506
1554/1563 [============================>.] - ETA: 0s - loss: 1.2813 - accuracy: 0.5505
1563/1563 [==============================] - 2s 1ms/step - loss: 1.2812 - accuracy: 0.5503
Epoch 10/10
1/1563 [..............................] - ETA: 2s - loss: 1.0484 - accuracy: 0.6250
39/1563 [..............................] - ETA: 2s - loss: 1.2836 - accuracy: 0.5633
77/1563 [>.............................] - ETA: 1s - loss: 1.2490 - accuracy: 0.5767
115/1563 [=>............................] - ETA: 1s - loss: 1.2608 - accuracy: 0.5679
153/1563 [=>............................] - ETA: 1s - loss: 1.2685 - accuracy: 0.5623
191/1563 [==>...........................] - ETA: 1s - loss: 1.2700 - accuracy: 0.5607
229/1563 [===>..........................] - ETA: 1s - loss: 1.2683 - accuracy: 0.5635
267/1563 [====>.........................] - ETA: 1s - loss: 1.2649 - accuracy: 0.5641
305/1563 [====>.........................] - ETA: 1s - loss: 1.2522 - accuracy: 0.5690
343/1563 [=====>........................] - ETA: 1s - loss: 1.2523 - accuracy: 0.5688
381/1563 [======>.......................] - ETA: 1s - loss: 1.2545 - accuracy: 0.5676
419/1563 [=======>......................] - ETA: 1s - loss: 1.2557 - accuracy: 0.5660
457/1563 [=======>......................] - ETA: 1s - loss: 1.2561 - accuracy: 0.5643
495/1563 [========>.....................] - ETA: 1s - loss: 1.2523 - accuracy: 0.5648
533/1563 [=========>....................] - ETA: 1s - loss: 1.2504 - accuracy: 0.5655
571/1563 [=========>....................] - ETA: 1s - loss: 1.2524 - accuracy: 0.5639
609/1563 [==========>...................] - ETA: 1s - loss: 1.2539 - accuracy: 0.5635
647/1563 [===========>..................] - ETA: 1s - loss: 1.2517 - accuracy: 0.5639
685/1563 [============>.................] - ETA: 1s - loss: 1.2517 - accuracy: 0.5630
723/1563 [============>.................] - ETA: 1s - loss: 1.2527 - accuracy: 0.5625
761/1563 [=============>................] - ETA: 1s - loss: 1.2553 - accuracy: 0.5604
799/1563 [==============>...............] - ETA: 1s - loss: 1.2538 - accuracy: 0.5612
837/1563 [===============>..............] - ETA: 0s - loss: 1.2525 - accuracy: 0.5617
875/1563 [===============>..............] - ETA: 0s - loss: 1.2510 - accuracy: 0.5624
913/1563 [================>.............] - ETA: 0s - loss: 1.2501 - accuracy: 0.5621
951/1563 [=================>............] - ETA: 0s - loss: 1.2506 - accuracy: 0.5620
989/1563 [=================>............] - ETA: 0s - loss: 1.2498 - accuracy: 0.5623
1027/1563 [==================>...........] - ETA: 0s - loss: 1.2519 - accuracy: 0.5620
1065/1563 [===================>..........] - ETA: 0s - loss: 1.2518 - accuracy: 0.5617
1103/1563 [====================>.........] - ETA: 0s - loss: 1.2521 - accuracy: 0.5620
1141/1563 [====================>.........] - ETA: 0s - loss: 1.2509 - accuracy: 0.5624
1179/1563 [=====================>........] - ETA: 0s - loss: 1.2519 - accuracy: 0.5622
1217/1563 [======================>.......] - ETA: 0s - loss: 1.2522 - accuracy: 0.5617
1255/1563 [=======================>......] - ETA: 0s - loss: 1.2514 - accuracy: 0.5620
1293/1563 [=======================>......] - ETA: 0s - loss: 1.2514 - accuracy: 0.5617
1331/1563 [========================>.....] - ETA: 0s - loss: 1.2522 - accuracy: 0.5611
1369/1563 [=========================>....] - ETA: 0s - loss: 1.2529 - accuracy: 0.5609
1407/1563 [==========================>...] - ETA: 0s - loss: 1.2517 - accuracy: 0.5612
1445/1563 [==========================>...] - ETA: 0s - loss: 1.2516 - accuracy: 0.5614
1482/1563 [===========================>..] - ETA: 0s - loss: 1.2527 - accuracy: 0.5609
1520/1563 [============================>.] - ETA: 0s - loss: 1.2524 - accuracy: 0.5608
1558/1563 [============================>.] - ETA: 0s - loss: 1.2524 - accuracy: 0.5607
1563/1563 [==============================] - 2s 1ms/step - loss: 1.2523 - accuracy: 0.5608
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
flatten (Flatten) (None, 3072) 0
dense (Dense) (None, 3000) 9219000
dense_1 (Dense) (None, 1000) 3001000
dense_2 (Dense) (None, 10) 10010
=================================================================
Total params: 12,230,010
Trainable params: 12,230,010
Non-trainable params: 0
_________________________________________________________________
None
wall time seconds (without loading test data set)= 37.12149000167847
hydro05
[SSH] completed
[SSH] exit-status: 0
[workspace] $ /bin/sh -xe /tmp/jenkins5904094894903426355.sh
+ scp 'HYDRO_REMOTE:~svchydrojenkins/python/tensorflow_time.txt' /var/lib/jenkins/jobs/tensorflow/workspace
Recording plot data
Saving plot series data from: /var/lib/jenkins/jobs/tensorflow/workspace/tensorflow_time.txt
Finished: SUCCESS
![[Jenkins]](/static/755cd4af/images/svgs/logo.svg){kind=logo}
