Started by timer
Running as SYSTEM
Building in workspace /var/lib/jenkins/jobs/tensorflow/workspace
[SSH] script:
module load anaconda3_gpu
module load cudnn
cd python
rm -f tensorflow_test.txt
srun --nodes=1 --ntasks-per-node=1 --mem=16g \
--time=00:05:00 \
--partition=a100rome \
--account=bbmb-hydro \
--gpus-per-node=1 \
--gpu-bind=closest \
python3 cifar10gpu.py | tee tensorflow_test.txt
YVALUE=`grep "wall time seconds" tensorflow_test.txt | cut -d'=' -f2`
echo YVALUE=$YVALUE > tensorflow_time.txt
[SSH] executing...
srun: job 96818 queued and waiting for resources
srun: job 96818 has been allocated resources
2024-11-06 07:43:07.065779: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-11-06 07:43:29.441240: E tensorflow/stream_executor/cuda/cuda_blas.cc:2981] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-11-06 07:43:59.962038: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /sw/spack/hydrogpu-2022-06/apps/cudnn/8.4.0.27-11.6-gcc-11.3.0-qbhsfav/lib:/sw/spack/hydrogpu-2022-06/apps/cuda/11.7.0-gcc-11.3.0-3ysno6b/lib64:/sw/spack/hydrogpu-2022-06/apps/openmpi/4.1.4-gcc-11.3.0-i6koipj/lib:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib64:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib
2024-11-06 07:43:59.962939: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /sw/spack/hydrogpu-2022-06/apps/cudnn/8.4.0.27-11.6-gcc-11.3.0-qbhsfav/lib:/sw/spack/hydrogpu-2022-06/apps/cuda/11.7.0-gcc-11.3.0-3ysno6b/lib64:/sw/spack/hydrogpu-2022-06/apps/openmpi/4.1.4-gcc-11.3.0-i6koipj/lib:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib64:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib
2024-11-06 07:43:59.962948: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
2024-11-06 07:44:45.518214: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-11-06 07:44:47.837224: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1616] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 79148 MB memory: -> device: 0, name: NVIDIA A100 80GB PCIe, pci bus id: 0000:21:00.0, compute capability: 8.0
(50000, 32, 32, 3) (10000, 32, 32, 3)
Epoch 1/10
2024-11-06 07:44:54.650887: I tensorflow/stream_executor/cuda/cuda_blas.cc:1614] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
1/1563 [..............................] - ETA: 1:33:57 - loss: 2.4187 - accuracy: 0.2188
37/1563 [..............................] - ETA: 2s - loss: 2.2785 - accuracy: 0.1596
75/1563 [>.............................] - ETA: 2s - loss: 2.2068 - accuracy: 0.1850
113/1563 [=>............................] - ETA: 1s - loss: 2.1624 - accuracy: 0.2022
151/1563 [=>............................] - ETA: 1s - loss: 2.1219 - accuracy: 0.2196
189/1563 [==>...........................] - ETA: 1s - loss: 2.0926 - accuracy: 0.2369
227/1563 [===>..........................] - ETA: 1s - loss: 2.0603 - accuracy: 0.2517
265/1563 [====>.........................] - ETA: 1s - loss: 2.0422 - accuracy: 0.2587
303/1563 [====>.........................] - ETA: 1s - loss: 2.0235 - accuracy: 0.2643
341/1563 [=====>........................] - ETA: 1s - loss: 2.0042 - accuracy: 0.2729
379/1563 [======>.......................] - ETA: 1s - loss: 1.9902 - accuracy: 0.2792
417/1563 [=======>......................] - ETA: 1s - loss: 1.9801 - accuracy: 0.2840
455/1563 [=======>......................] - ETA: 1s - loss: 1.9702 - accuracy: 0.2885
493/1563 [========>.....................] - ETA: 1s - loss: 1.9578 - accuracy: 0.2946
531/1563 [=========>....................] - ETA: 1s - loss: 1.9485 - accuracy: 0.2982
569/1563 [=========>....................] - ETA: 1s - loss: 1.9411 - accuracy: 0.3017
607/1563 [==========>...................] - ETA: 1s - loss: 1.9335 - accuracy: 0.3046
645/1563 [===========>..................] - ETA: 1s - loss: 1.9256 - accuracy: 0.3079
683/1563 [============>.................] - ETA: 1s - loss: 1.9176 - accuracy: 0.3115
721/1563 [============>.................] - ETA: 1s - loss: 1.9111 - accuracy: 0.3141
759/1563 [=============>................] - ETA: 1s - loss: 1.9056 - accuracy: 0.3167
797/1563 [==============>...............] - ETA: 1s - loss: 1.9010 - accuracy: 0.3193
835/1563 [===============>..............] - ETA: 0s - loss: 1.8958 - accuracy: 0.3214
873/1563 [===============>..............] - ETA: 0s - loss: 1.8892 - accuracy: 0.3241
911/1563 [================>.............] - ETA: 0s - loss: 1.8813 - accuracy: 0.3271
949/1563 [=================>............] - ETA: 0s - loss: 1.8765 - accuracy: 0.3292
987/1563 [=================>............] - ETA: 0s - loss: 1.8719 - accuracy: 0.3311
1025/1563 [==================>...........] - ETA: 0s - loss: 1.8666 - accuracy: 0.3330
1063/1563 [===================>..........] - ETA: 0s - loss: 1.8625 - accuracy: 0.3349
1101/1563 [====================>.........] - ETA: 0s - loss: 1.8566 - accuracy: 0.3372
1139/1563 [====================>.........] - ETA: 0s - loss: 1.8535 - accuracy: 0.3380
1177/1563 [=====================>........] - ETA: 0s - loss: 1.8510 - accuracy: 0.3395
1215/1563 [======================>.......] - ETA: 0s - loss: 1.8455 - accuracy: 0.3415
1253/1563 [=======================>......] - ETA: 0s - loss: 1.8416 - accuracy: 0.3423
1291/1563 [=======================>......] - ETA: 0s - loss: 1.8389 - accuracy: 0.3431
1329/1563 [========================>.....] - ETA: 0s - loss: 1.8356 - accuracy: 0.3444
1367/1563 [=========================>....] - ETA: 0s - loss: 1.8315 - accuracy: 0.3460
1405/1563 [=========================>....] - ETA: 0s - loss: 1.8282 - accuracy: 0.3477
1443/1563 [==========================>...] - ETA: 0s - loss: 1.8245 - accuracy: 0.3489
1481/1563 [===========================>..] - ETA: 0s - loss: 1.8210 - accuracy: 0.3502
1519/1563 [============================>.] - ETA: 0s - loss: 1.8189 - accuracy: 0.3513
1557/1563 [============================>.] - ETA: 0s - loss: 1.8165 - accuracy: 0.3522
1563/1563 [==============================] - 6s 1ms/step - loss: 1.8162 - accuracy: 0.3521
Epoch 2/10
1/1563 [..............................] - ETA: 2s - loss: 1.5878 - accuracy: 0.3750
39/1563 [..............................] - ETA: 2s - loss: 1.6656 - accuracy: 0.4095
77/1563 [>.............................] - ETA: 1s - loss: 1.6609 - accuracy: 0.4180
115/1563 [=>............................] - ETA: 1s - loss: 1.6704 - accuracy: 0.4141
153/1563 [=>............................] - ETA: 1s - loss: 1.6672 - accuracy: 0.4150
191/1563 [==>...........................] - ETA: 1s - loss: 1.6682 - accuracy: 0.4143
230/1563 [===>..........................] - ETA: 1s - loss: 1.6671 - accuracy: 0.4129
268/1563 [====>.........................] - ETA: 1s - loss: 1.6613 - accuracy: 0.4128
306/1563 [====>.........................] - ETA: 1s - loss: 1.6553 - accuracy: 0.4137
344/1563 [=====>........................] - ETA: 1s - loss: 1.6574 - accuracy: 0.4141
382/1563 [======>.......................] - ETA: 1s - loss: 1.6558 - accuracy: 0.4171
420/1563 [=======>......................] - ETA: 1s - loss: 1.6563 - accuracy: 0.4161
458/1563 [=======>......................] - ETA: 1s - loss: 1.6569 - accuracy: 0.4152
496/1563 [========>.....................] - ETA: 1s - loss: 1.6572 - accuracy: 0.4144
534/1563 [=========>....................] - ETA: 1s - loss: 1.6532 - accuracy: 0.4161
572/1563 [=========>....................] - ETA: 1s - loss: 1.6525 - accuracy: 0.4162
610/1563 [==========>...................] - ETA: 1s - loss: 1.6507 - accuracy: 0.4165
648/1563 [===========>..................] - ETA: 1s - loss: 1.6480 - accuracy: 0.4183
686/1563 [============>.................] - ETA: 1s - loss: 1.6445 - accuracy: 0.4198
724/1563 [============>.................] - ETA: 1s - loss: 1.6441 - accuracy: 0.4194
762/1563 [=============>................] - ETA: 1s - loss: 1.6426 - accuracy: 0.4195
800/1563 [==============>...............] - ETA: 1s - loss: 1.6420 - accuracy: 0.4198
838/1563 [===============>..............] - ETA: 0s - loss: 1.6417 - accuracy: 0.4196
876/1563 [===============>..............] - ETA: 0s - loss: 1.6411 - accuracy: 0.4201
914/1563 [================>.............] - ETA: 0s - loss: 1.6400 - accuracy: 0.4211
952/1563 [=================>............] - ETA: 0s - loss: 1.6390 - accuracy: 0.4213
990/1563 [==================>...........] - ETA: 0s - loss: 1.6377 - accuracy: 0.4217
1028/1563 [==================>...........] - ETA: 0s - loss: 1.6364 - accuracy: 0.4224
1066/1563 [===================>..........] - ETA: 0s - loss: 1.6352 - accuracy: 0.4229
1104/1563 [====================>.........] - ETA: 0s - loss: 1.6352 - accuracy: 0.4237
1142/1563 [====================>.........] - ETA: 0s - loss: 1.6350 - accuracy: 0.4236
1180/1563 [=====================>........] - ETA: 0s - loss: 1.6335 - accuracy: 0.4239
1218/1563 [======================>.......] - ETA: 0s - loss: 1.6335 - accuracy: 0.4239
1256/1563 [=======================>......] - ETA: 0s - loss: 1.6327 - accuracy: 0.4241
1294/1563 [=======================>......] - ETA: 0s - loss: 1.6311 - accuracy: 0.4242
1332/1563 [========================>.....] - ETA: 0s - loss: 1.6306 - accuracy: 0.4245
1370/1563 [=========================>....] - ETA: 0s - loss: 1.6292 - accuracy: 0.4249
1408/1563 [==========================>...] - ETA: 0s - loss: 1.6281 - accuracy: 0.4249
1446/1563 [==========================>...] - ETA: 0s - loss: 1.6271 - accuracy: 0.4252
1484/1563 [===========================>..] - ETA: 0s - loss: 1.6254 - accuracy: 0.4261
1522/1563 [============================>.] - ETA: 0s - loss: 1.6246 - accuracy: 0.4263
1560/1563 [============================>.] - ETA: 0s - loss: 1.6238 - accuracy: 0.4265
1563/1563 [==============================] - 2s 1ms/step - loss: 1.6239 - accuracy: 0.4265
Epoch 3/10
1/1563 [..............................] - ETA: 2s - loss: 1.3312 - accuracy: 0.5625
39/1563 [..............................] - ETA: 2s - loss: 1.5166 - accuracy: 0.4527
77/1563 [>.............................] - ETA: 1s - loss: 1.5258 - accuracy: 0.4570
115/1563 [=>............................] - ETA: 1s - loss: 1.5455 - accuracy: 0.4543
153/1563 [=>............................] - ETA: 1s - loss: 1.5424 - accuracy: 0.4569
191/1563 [==>...........................] - ETA: 1s - loss: 1.5524 - accuracy: 0.4521
229/1563 [===>..........................] - ETA: 1s - loss: 1.5587 - accuracy: 0.4498
267/1563 [====>.........................] - ETA: 1s - loss: 1.5551 - accuracy: 0.4494
305/1563 [====>.........................] - ETA: 1s - loss: 1.5556 - accuracy: 0.4497
343/1563 [=====>........................] - ETA: 1s - loss: 1.5611 - accuracy: 0.4462
381/1563 [======>.......................] - ETA: 1s - loss: 1.5621 - accuracy: 0.4451
419/1563 [=======>......................] - ETA: 1s - loss: 1.5642 - accuracy: 0.4442
457/1563 [=======>......................] - ETA: 1s - loss: 1.5643 - accuracy: 0.4447
495/1563 [========>.....................] - ETA: 1s - loss: 1.5652 - accuracy: 0.4446
533/1563 [=========>....................] - ETA: 1s - loss: 1.5663 - accuracy: 0.4445
571/1563 [=========>....................] - ETA: 1s - loss: 1.5631 - accuracy: 0.4464
610/1563 [==========>...................] - ETA: 1s - loss: 1.5613 - accuracy: 0.4472
648/1563 [===========>..................] - ETA: 1s - loss: 1.5622 - accuracy: 0.4464
686/1563 [============>.................] - ETA: 1s - loss: 1.5602 - accuracy: 0.4478
724/1563 [============>.................] - ETA: 1s - loss: 1.5599 - accuracy: 0.4474
762/1563 [=============>................] - ETA: 1s - loss: 1.5607 - accuracy: 0.4470
800/1563 [==============>...............] - ETA: 1s - loss: 1.5589 - accuracy: 0.4477
837/1563 [===============>..............] - ETA: 0s - loss: 1.5579 - accuracy: 0.4475
875/1563 [===============>..............] - ETA: 0s - loss: 1.5567 - accuracy: 0.4478
913/1563 [================>.............] - ETA: 0s - loss: 1.5551 - accuracy: 0.4493
951/1563 [=================>............] - ETA: 0s - loss: 1.5543 - accuracy: 0.4496
989/1563 [=================>............] - ETA: 0s - loss: 1.5543 - accuracy: 0.4492
1027/1563 [==================>...........] - ETA: 0s - loss: 1.5525 - accuracy: 0.4501
1065/1563 [===================>..........] - ETA: 0s - loss: 1.5513 - accuracy: 0.4507
1103/1563 [====================>.........] - ETA: 0s - loss: 1.5516 - accuracy: 0.4509
1141/1563 [====================>.........] - ETA: 0s - loss: 1.5523 - accuracy: 0.4507
1179/1563 [=====================>........] - ETA: 0s - loss: 1.5508 - accuracy: 0.4513
1217/1563 [======================>.......] - ETA: 0s - loss: 1.5499 - accuracy: 0.4515
1255/1563 [=======================>......] - ETA: 0s - loss: 1.5503 - accuracy: 0.4517
1293/1563 [=======================>......] - ETA: 0s - loss: 1.5497 - accuracy: 0.4522
1331/1563 [========================>.....] - ETA: 0s - loss: 1.5482 - accuracy: 0.4526
1369/1563 [=========================>....] - ETA: 0s - loss: 1.5487 - accuracy: 0.4525
1407/1563 [==========================>...] - ETA: 0s - loss: 1.5475 - accuracy: 0.4529
1445/1563 [==========================>...] - ETA: 0s - loss: 1.5474 - accuracy: 0.4529
1483/1563 [===========================>..] - ETA: 0s - loss: 1.5454 - accuracy: 0.4533
1521/1563 [============================>.] - ETA: 0s - loss: 1.5442 - accuracy: 0.4536
1559/1563 [============================>.] - ETA: 0s - loss: 1.5430 - accuracy: 0.4540
1563/1563 [==============================] - 2s 1ms/step - loss: 1.5429 - accuracy: 0.4541
Epoch 4/10
1/1563 [..............................] - ETA: 2s - loss: 1.4669 - accuracy: 0.4375
39/1563 [..............................] - ETA: 2s - loss: 1.5227 - accuracy: 0.4607
75/1563 [>.............................] - ETA: 2s - loss: 1.5003 - accuracy: 0.4704
113/1563 [=>............................] - ETA: 1s - loss: 1.5002 - accuracy: 0.4776
151/1563 [=>............................] - ETA: 1s - loss: 1.5008 - accuracy: 0.4741
189/1563 [==>...........................] - ETA: 1s - loss: 1.4981 - accuracy: 0.4740
227/1563 [===>..........................] - ETA: 1s - loss: 1.5011 - accuracy: 0.4711
265/1563 [====>.........................] - ETA: 1s - loss: 1.5012 - accuracy: 0.4729
303/1563 [====>.........................] - ETA: 1s - loss: 1.5068 - accuracy: 0.4698
341/1563 [=====>........................] - ETA: 1s - loss: 1.5047 - accuracy: 0.4716
379/1563 [======>.......................] - ETA: 1s - loss: 1.5062 - accuracy: 0.4704
417/1563 [=======>......................] - ETA: 1s - loss: 1.4998 - accuracy: 0.4732
455/1563 [=======>......................] - ETA: 1s - loss: 1.5012 - accuracy: 0.4725
493/1563 [========>.....................] - ETA: 1s - loss: 1.4987 - accuracy: 0.4739
531/1563 [=========>....................] - ETA: 1s - loss: 1.4968 - accuracy: 0.4756
569/1563 [=========>....................] - ETA: 1s - loss: 1.4957 - accuracy: 0.4763
607/1563 [==========>...................] - ETA: 1s - loss: 1.4931 - accuracy: 0.4770
645/1563 [===========>..................] - ETA: 1s - loss: 1.4938 - accuracy: 0.4758
683/1563 [============>.................] - ETA: 1s - loss: 1.4916 - accuracy: 0.4770
721/1563 [============>.................] - ETA: 1s - loss: 1.4927 - accuracy: 0.4762
759/1563 [=============>................] - ETA: 1s - loss: 1.4928 - accuracy: 0.4766
797/1563 [==============>...............] - ETA: 1s - loss: 1.4923 - accuracy: 0.4773
835/1563 [===============>..............] - ETA: 0s - loss: 1.4905 - accuracy: 0.4781
873/1563 [===============>..............] - ETA: 0s - loss: 1.4895 - accuracy: 0.4779
911/1563 [================>.............] - ETA: 0s - loss: 1.4892 - accuracy: 0.4779
949/1563 [=================>............] - ETA: 0s - loss: 1.4890 - accuracy: 0.4781
987/1563 [=================>............] - ETA: 0s - loss: 1.4894 - accuracy: 0.4774
1025/1563 [==================>...........] - ETA: 0s - loss: 1.4888 - accuracy: 0.4773
1063/1563 [===================>..........] - ETA: 0s - loss: 1.4900 - accuracy: 0.4764
1101/1563 [====================>.........] - ETA: 0s - loss: 1.4887 - accuracy: 0.4762
1139/1563 [====================>.........] - ETA: 0s - loss: 1.4856 - accuracy: 0.4777
1177/1563 [=====================>........] - ETA: 0s - loss: 1.4844 - accuracy: 0.4775
1215/1563 [======================>.......] - ETA: 0s - loss: 1.4848 - accuracy: 0.4776
1253/1563 [=======================>......] - ETA: 0s - loss: 1.4851 - accuracy: 0.4781
1291/1563 [=======================>......] - ETA: 0s - loss: 1.4837 - accuracy: 0.4784
1329/1563 [========================>.....] - ETA: 0s - loss: 1.4832 - accuracy: 0.4787
1367/1563 [=========================>....] - ETA: 0s - loss: 1.4833 - accuracy: 0.4784
1405/1563 [=========================>....] - ETA: 0s - loss: 1.4821 - accuracy: 0.4788
1443/1563 [==========================>...] - ETA: 0s - loss: 1.4811 - accuracy: 0.4789
1481/1563 [===========================>..] - ETA: 0s - loss: 1.4806 - accuracy: 0.4790
1519/1563 [============================>.] - ETA: 0s - loss: 1.4807 - accuracy: 0.4787
1557/1563 [============================>.] - ETA: 0s - loss: 1.4803 - accuracy: 0.4789
1563/1563 [==============================] - 2s 1ms/step - loss: 1.4803 - accuracy: 0.4789
Epoch 5/10
1/1563 [..............................] - ETA: 2s - loss: 1.4584 - accuracy: 0.4688
39/1563 [..............................] - ETA: 2s - loss: 1.4641 - accuracy: 0.4720
77/1563 [>.............................] - ETA: 1s - loss: 1.4600 - accuracy: 0.4821
115/1563 [=>............................] - ETA: 1s - loss: 1.4492 - accuracy: 0.4870
153/1563 [=>............................] - ETA: 1s - loss: 1.4538 - accuracy: 0.4871
192/1563 [==>...........................] - ETA: 1s - loss: 1.4533 - accuracy: 0.4876
230/1563 [===>..........................] - ETA: 1s - loss: 1.4541 - accuracy: 0.4834
268/1563 [====>.........................] - ETA: 1s - loss: 1.4466 - accuracy: 0.4883
306/1563 [====>.........................] - ETA: 1s - loss: 1.4480 - accuracy: 0.4885
344/1563 [=====>........................] - ETA: 1s - loss: 1.4489 - accuracy: 0.4886
382/1563 [======>.......................] - ETA: 1s - loss: 1.4484 - accuracy: 0.4899
420/1563 [=======>......................] - ETA: 1s - loss: 1.4465 - accuracy: 0.4899
458/1563 [=======>......................] - ETA: 1s - loss: 1.4425 - accuracy: 0.4907
496/1563 [========>.....................] - ETA: 1s - loss: 1.4410 - accuracy: 0.4921
534/1563 [=========>....................] - ETA: 1s - loss: 1.4407 - accuracy: 0.4912
572/1563 [=========>....................] - ETA: 1s - loss: 1.4401 - accuracy: 0.4915
610/1563 [==========>...................] - ETA: 1s - loss: 1.4389 - accuracy: 0.4912
648/1563 [===========>..................] - ETA: 1s - loss: 1.4375 - accuracy: 0.4918
686/1563 [============>.................] - ETA: 1s - loss: 1.4372 - accuracy: 0.4929
724/1563 [============>.................] - ETA: 1s - loss: 1.4360 - accuracy: 0.4931
762/1563 [=============>................] - ETA: 1s - loss: 1.4353 - accuracy: 0.4934
800/1563 [==============>...............] - ETA: 1s - loss: 1.4367 - accuracy: 0.4929
838/1563 [===============>..............] - ETA: 0s - loss: 1.4379 - accuracy: 0.4923
876/1563 [===============>..............] - ETA: 0s - loss: 1.4384 - accuracy: 0.4916
914/1563 [================>.............] - ETA: 0s - loss: 1.4365 - accuracy: 0.4920
952/1563 [=================>............] - ETA: 0s - loss: 1.4370 - accuracy: 0.4924
990/1563 [==================>...........] - ETA: 0s - loss: 1.4381 - accuracy: 0.4918
1028/1563 [==================>...........] - ETA: 0s - loss: 1.4365 - accuracy: 0.4932
1066/1563 [===================>..........] - ETA: 0s - loss: 1.4369 - accuracy: 0.4933
1104/1563 [====================>.........] - ETA: 0s - loss: 1.4386 - accuracy: 0.4927
1142/1563 [====================>.........] - ETA: 0s - loss: 1.4365 - accuracy: 0.4935
1180/1563 [=====================>........] - ETA: 0s - loss: 1.4341 - accuracy: 0.4944
1218/1563 [======================>.......] - ETA: 0s - loss: 1.4348 - accuracy: 0.4940
1256/1563 [=======================>......] - ETA: 0s - loss: 1.4339 - accuracy: 0.4942
1294/1563 [=======================>......] - ETA: 0s - loss: 1.4346 - accuracy: 0.4942
1332/1563 [========================>.....] - ETA: 0s - loss: 1.4348 - accuracy: 0.4942
1370/1563 [=========================>....] - ETA: 0s - loss: 1.4336 - accuracy: 0.4945
1408/1563 [==========================>...] - ETA: 0s - loss: 1.4344 - accuracy: 0.4947
1446/1563 [==========================>...] - ETA: 0s - loss: 1.4333 - accuracy: 0.4949
1484/1563 [===========================>..] - ETA: 0s - loss: 1.4325 - accuracy: 0.4952
1523/1563 [============================>.] - ETA: 0s - loss: 1.4325 - accuracy: 0.4957
1561/1563 [============================>.] - ETA: 0s - loss: 1.4330 - accuracy: 0.4958
1563/1563 [==============================] - 2s 1ms/step - loss: 1.4331 - accuracy: 0.4958
Epoch 6/10
1/1563 [..............................] - ETA: 2s - loss: 1.5581 - accuracy: 0.4688
39/1563 [..............................] - ETA: 2s - loss: 1.4124 - accuracy: 0.5056
77/1563 [>.............................] - ETA: 1s - loss: 1.3802 - accuracy: 0.5138
115/1563 [=>............................] - ETA: 1s - loss: 1.3774 - accuracy: 0.5168
153/1563 [=>............................] - ETA: 1s - loss: 1.3721 - accuracy: 0.5188
192/1563 [==>...........................] - ETA: 1s - loss: 1.3798 - accuracy: 0.5166
231/1563 [===>..........................] - ETA: 1s - loss: 1.3762 - accuracy: 0.5193
270/1563 [====>.........................] - ETA: 1s - loss: 1.3820 - accuracy: 0.5139
309/1563 [====>.........................] - ETA: 1s - loss: 1.3774 - accuracy: 0.5183
348/1563 [=====>........................] - ETA: 1s - loss: 1.3783 - accuracy: 0.5175
386/1563 [======>.......................] - ETA: 1s - loss: 1.3773 - accuracy: 0.5176
424/1563 [=======>......................] - ETA: 1s - loss: 1.3789 - accuracy: 0.5158
463/1563 [=======>......................] - ETA: 1s - loss: 1.3789 - accuracy: 0.5167
501/1563 [========>.....................] - ETA: 1s - loss: 1.3802 - accuracy: 0.5162
539/1563 [=========>....................] - ETA: 1s - loss: 1.3837 - accuracy: 0.5141
578/1563 [==========>...................] - ETA: 1s - loss: 1.3827 - accuracy: 0.5145
617/1563 [==========>...................] - ETA: 1s - loss: 1.3828 - accuracy: 0.5151
656/1563 [===========>..................] - ETA: 1s - loss: 1.3849 - accuracy: 0.5147
694/1563 [============>.................] - ETA: 1s - loss: 1.3879 - accuracy: 0.5137
733/1563 [=============>................] - ETA: 1s - loss: 1.3870 - accuracy: 0.5134
772/1563 [=============>................] - ETA: 1s - loss: 1.3894 - accuracy: 0.5120
810/1563 [==============>...............] - ETA: 0s - loss: 1.3885 - accuracy: 0.5128
848/1563 [===============>..............] - ETA: 0s - loss: 1.3906 - accuracy: 0.5119
887/1563 [================>.............] - ETA: 0s - loss: 1.3900 - accuracy: 0.5122
926/1563 [================>.............] - ETA: 0s - loss: 1.3895 - accuracy: 0.5123
964/1563 [=================>............] - ETA: 0s - loss: 1.3878 - accuracy: 0.5129
1002/1563 [==================>...........] - ETA: 0s - loss: 1.3878 - accuracy: 0.5128
1040/1563 [==================>...........] - ETA: 0s - loss: 1.3876 - accuracy: 0.5127
1079/1563 [===================>..........] - ETA: 0s - loss: 1.3887 - accuracy: 0.5125
1118/1563 [====================>.........] - ETA: 0s - loss: 1.3887 - accuracy: 0.5126
1157/1563 [=====================>........] - ETA: 0s - loss: 1.3897 - accuracy: 0.5129
1195/1563 [=====================>........] - ETA: 0s - loss: 1.3895 - accuracy: 0.5132
1233/1563 [======================>.......] - ETA: 0s - loss: 1.3915 - accuracy: 0.5124
1272/1563 [=======================>......] - ETA: 0s - loss: 1.3922 - accuracy: 0.5123
1310/1563 [========================>.....] - ETA: 0s - loss: 1.3920 - accuracy: 0.5120
1348/1563 [========================>.....] - ETA: 0s - loss: 1.3920 - accuracy: 0.5115
1387/1563 [=========================>....] - ETA: 0s - loss: 1.3911 - accuracy: 0.5116
1425/1563 [==========================>...] - ETA: 0s - loss: 1.3908 - accuracy: 0.5118
1464/1563 [===========================>..] - ETA: 0s - loss: 1.3898 - accuracy: 0.5120
1502/1563 [===========================>..] - ETA: 0s - loss: 1.3899 - accuracy: 0.5118
1540/1563 [============================>.] - ETA: 0s - loss: 1.3891 - accuracy: 0.5119
1563/1563 [==============================] - 2s 1ms/step - loss: 1.3905 - accuracy: 0.5111
Epoch 7/10
1/1563 [..............................] - ETA: 2s - loss: 1.5738 - accuracy: 0.4688
39/1563 [..............................] - ETA: 2s - loss: 1.3544 - accuracy: 0.5521
78/1563 [>.............................] - ETA: 1s - loss: 1.3478 - accuracy: 0.5465
117/1563 [=>............................] - ETA: 1s - loss: 1.3431 - accuracy: 0.5441
155/1563 [=>............................] - ETA: 1s - loss: 1.3489 - accuracy: 0.5415
194/1563 [==>...........................] - ETA: 1s - loss: 1.3451 - accuracy: 0.5390
233/1563 [===>..........................] - ETA: 1s - loss: 1.3409 - accuracy: 0.5374
271/1563 [====>.........................] - ETA: 1s - loss: 1.3423 - accuracy: 0.5357
309/1563 [====>.........................] - ETA: 1s - loss: 1.3426 - accuracy: 0.5342
347/1563 [=====>........................] - ETA: 1s - loss: 1.3439 - accuracy: 0.5327
386/1563 [======>.......................] - ETA: 1s - loss: 1.3489 - accuracy: 0.5304
425/1563 [=======>......................] - ETA: 1s - loss: 1.3524 - accuracy: 0.5290
464/1563 [=======>......................] - ETA: 1s - loss: 1.3534 - accuracy: 0.5273
503/1563 [========>.....................] - ETA: 1s - loss: 1.3555 - accuracy: 0.5262
541/1563 [=========>....................] - ETA: 1s - loss: 1.3567 - accuracy: 0.5256
580/1563 [==========>...................] - ETA: 1s - loss: 1.3584 - accuracy: 0.5239
619/1563 [==========>...................] - ETA: 1s - loss: 1.3576 - accuracy: 0.5239
657/1563 [===========>..................] - ETA: 1s - loss: 1.3598 - accuracy: 0.5231
695/1563 [============>.................] - ETA: 1s - loss: 1.3600 - accuracy: 0.5238
734/1563 [=============>................] - ETA: 1s - loss: 1.3587 - accuracy: 0.5252
773/1563 [=============>................] - ETA: 1s - loss: 1.3607 - accuracy: 0.5248
811/1563 [==============>...............] - ETA: 0s - loss: 1.3595 - accuracy: 0.5253
849/1563 [===============>..............] - ETA: 0s - loss: 1.3595 - accuracy: 0.5250
888/1563 [================>.............] - ETA: 0s - loss: 1.3589 - accuracy: 0.5251
926/1563 [================>.............] - ETA: 0s - loss: 1.3578 - accuracy: 0.5249
964/1563 [=================>............] - ETA: 0s - loss: 1.3570 - accuracy: 0.5253
1002/1563 [==================>...........] - ETA: 0s - loss: 1.3577 - accuracy: 0.5252
1041/1563 [==================>...........] - ETA: 0s - loss: 1.3565 - accuracy: 0.5254
1080/1563 [===================>..........] - ETA: 0s - loss: 1.3563 - accuracy: 0.5253
1119/1563 [====================>.........] - ETA: 0s - loss: 1.3561 - accuracy: 0.5255
1157/1563 [=====================>........] - ETA: 0s - loss: 1.3576 - accuracy: 0.5251
1195/1563 [=====================>........] - ETA: 0s - loss: 1.3568 - accuracy: 0.5254
1233/1563 [======================>.......] - ETA: 0s - loss: 1.3554 - accuracy: 0.5260
1271/1563 [=======================>......] - ETA: 0s - loss: 1.3550 - accuracy: 0.5262
1309/1563 [========================>.....] - ETA: 0s - loss: 1.3554 - accuracy: 0.5260
1347/1563 [========================>.....] - ETA: 0s - loss: 1.3545 - accuracy: 0.5267
1386/1563 [=========================>....] - ETA: 0s - loss: 1.3537 - accuracy: 0.5268
1425/1563 [==========================>...] - ETA: 0s - loss: 1.3535 - accuracy: 0.5269
1463/1563 [===========================>..] - ETA: 0s - loss: 1.3530 - accuracy: 0.5274
1501/1563 [===========================>..] - ETA: 0s - loss: 1.3531 - accuracy: 0.5270
1540/1563 [============================>.] - ETA: 0s - loss: 1.3524 - accuracy: 0.5271
1563/1563 [==============================] - 2s 1ms/step - loss: 1.3526 - accuracy: 0.5270
Epoch 8/10
1/1563 [..............................] - ETA: 1s - loss: 1.0286 - accuracy: 0.6250
39/1563 [..............................] - ETA: 2s - loss: 1.2688 - accuracy: 0.5489
78/1563 [>.............................] - ETA: 1s - loss: 1.3165 - accuracy: 0.5337
117/1563 [=>............................] - ETA: 1s - loss: 1.3086 - accuracy: 0.5417
156/1563 [=>............................] - ETA: 1s - loss: 1.3090 - accuracy: 0.5413
194/1563 [==>...........................] - ETA: 1s - loss: 1.3086 - accuracy: 0.5374
232/1563 [===>..........................] - ETA: 1s - loss: 1.3155 - accuracy: 0.5331
270/1563 [====>.........................] - ETA: 1s - loss: 1.3197 - accuracy: 0.5324
308/1563 [====>.........................] - ETA: 1s - loss: 1.3225 - accuracy: 0.5332
346/1563 [=====>........................] - ETA: 1s - loss: 1.3265 - accuracy: 0.5304
385/1563 [======>.......................] - ETA: 1s - loss: 1.3272 - accuracy: 0.5304
424/1563 [=======>......................] - ETA: 1s - loss: 1.3302 - accuracy: 0.5286
463/1563 [=======>......................] - ETA: 1s - loss: 1.3286 - accuracy: 0.5315
502/1563 [========>.....................] - ETA: 1s - loss: 1.3296 - accuracy: 0.5317
540/1563 [=========>....................] - ETA: 1s - loss: 1.3312 - accuracy: 0.5312
578/1563 [==========>...................] - ETA: 1s - loss: 1.3307 - accuracy: 0.5314
616/1563 [==========>...................] - ETA: 1s - loss: 1.3287 - accuracy: 0.5322
655/1563 [===========>..................] - ETA: 1s - loss: 1.3295 - accuracy: 0.5317
694/1563 [============>.................] - ETA: 1s - loss: 1.3283 - accuracy: 0.5317
732/1563 [=============>................] - ETA: 1s - loss: 1.3289 - accuracy: 0.5315
771/1563 [=============>................] - ETA: 1s - loss: 1.3262 - accuracy: 0.5327
809/1563 [==============>...............] - ETA: 0s - loss: 1.3249 - accuracy: 0.5332
847/1563 [===============>..............] - ETA: 0s - loss: 1.3244 - accuracy: 0.5335
885/1563 [===============>..............] - ETA: 0s - loss: 1.3210 - accuracy: 0.5347
923/1563 [================>.............] - ETA: 0s - loss: 1.3201 - accuracy: 0.5348
961/1563 [=================>............] - ETA: 0s - loss: 1.3204 - accuracy: 0.5347
1000/1563 [==================>...........] - ETA: 0s - loss: 1.3204 - accuracy: 0.5345
1039/1563 [==================>...........] - ETA: 0s - loss: 1.3190 - accuracy: 0.5351
1078/1563 [===================>..........] - ETA: 0s - loss: 1.3189 - accuracy: 0.5349
1116/1563 [====================>.........] - ETA: 0s - loss: 1.3198 - accuracy: 0.5346
1155/1563 [=====================>........] - ETA: 0s - loss: 1.3196 - accuracy: 0.5349
1193/1563 [=====================>........] - ETA: 0s - loss: 1.3196 - accuracy: 0.5350
1231/1563 [======================>.......] - ETA: 0s - loss: 1.3189 - accuracy: 0.5351
1270/1563 [=======================>......] - ETA: 0s - loss: 1.3183 - accuracy: 0.5351
1309/1563 [========================>.....] - ETA: 0s - loss: 1.3177 - accuracy: 0.5351
1348/1563 [========================>.....] - ETA: 0s - loss: 1.3177 - accuracy: 0.5352
1386/1563 [=========================>....] - ETA: 0s - loss: 1.3172 - accuracy: 0.5354
1425/1563 [==========================>...] - ETA: 0s - loss: 1.3176 - accuracy: 0.5350
1463/1563 [===========================>..] - ETA: 0s - loss: 1.3182 - accuracy: 0.5352
1501/1563 [===========================>..] - ETA: 0s - loss: 1.3185 - accuracy: 0.5348
1540/1563 [============================>.] - ETA: 0s - loss: 1.3185 - accuracy: 0.5351
1563/1563 [==============================] - 2s 1ms/step - loss: 1.3182 - accuracy: 0.5356
Epoch 9/10
1/1563 [..............................] - ETA: 2s - loss: 1.3349 - accuracy: 0.5312
40/1563 [..............................] - ETA: 1s - loss: 1.2929 - accuracy: 0.5641
79/1563 [>.............................] - ETA: 1s - loss: 1.2668 - accuracy: 0.5688
118/1563 [=>............................] - ETA: 1s - loss: 1.2741 - accuracy: 0.5665
157/1563 [==>...........................] - ETA: 1s - loss: 1.2741 - accuracy: 0.5621
195/1563 [==>...........................] - ETA: 1s - loss: 1.2668 - accuracy: 0.5636
233/1563 [===>..........................] - ETA: 1s - loss: 1.2659 - accuracy: 0.5622
271/1563 [====>.........................] - ETA: 1s - loss: 1.2745 - accuracy: 0.5574
309/1563 [====>.........................] - ETA: 1s - loss: 1.2740 - accuracy: 0.5566
348/1563 [=====>........................] - ETA: 1s - loss: 1.2751 - accuracy: 0.5539
387/1563 [======>.......................] - ETA: 1s - loss: 1.2782 - accuracy: 0.5529
426/1563 [=======>......................] - ETA: 1s - loss: 1.2840 - accuracy: 0.5507
464/1563 [=======>......................] - ETA: 1s - loss: 1.2860 - accuracy: 0.5502
502/1563 [========>.....................] - ETA: 1s - loss: 1.2846 - accuracy: 0.5502
540/1563 [=========>....................] - ETA: 1s - loss: 1.2824 - accuracy: 0.5509
579/1563 [==========>...................] - ETA: 1s - loss: 1.2824 - accuracy: 0.5495
617/1563 [==========>...................] - ETA: 1s - loss: 1.2803 - accuracy: 0.5497
656/1563 [===========>..................] - ETA: 1s - loss: 1.2790 - accuracy: 0.5501
694/1563 [============>.................] - ETA: 1s - loss: 1.2809 - accuracy: 0.5502
732/1563 [=============>................] - ETA: 1s - loss: 1.2801 - accuracy: 0.5508
771/1563 [=============>................] - ETA: 1s - loss: 1.2793 - accuracy: 0.5514
809/1563 [==============>...............] - ETA: 0s - loss: 1.2782 - accuracy: 0.5518
847/1563 [===============>..............] - ETA: 0s - loss: 1.2798 - accuracy: 0.5518
886/1563 [================>.............] - ETA: 0s - loss: 1.2811 - accuracy: 0.5511
925/1563 [================>.............] - ETA: 0s - loss: 1.2820 - accuracy: 0.5509
964/1563 [=================>............] - ETA: 0s - loss: 1.2809 - accuracy: 0.5510
1003/1563 [==================>...........] - ETA: 0s - loss: 1.2804 - accuracy: 0.5514
1041/1563 [==================>...........] - ETA: 0s - loss: 1.2816 - accuracy: 0.5509
1079/1563 [===================>..........] - ETA: 0s - loss: 1.2820 - accuracy: 0.5513
1117/1563 [====================>.........] - ETA: 0s - loss: 1.2834 - accuracy: 0.5507
1155/1563 [=====================>........] - ETA: 0s - loss: 1.2820 - accuracy: 0.5512
1193/1563 [=====================>........] - ETA: 0s - loss: 1.2830 - accuracy: 0.5517
1231/1563 [======================>.......] - ETA: 0s - loss: 1.2829 - accuracy: 0.5519
1270/1563 [=======================>......] - ETA: 0s - loss: 1.2833 - accuracy: 0.5518
1308/1563 [========================>.....] - ETA: 0s - loss: 1.2835 - accuracy: 0.5512
1346/1563 [========================>.....] - ETA: 0s - loss: 1.2844 - accuracy: 0.5506
1384/1563 [=========================>....] - ETA: 0s - loss: 1.2840 - accuracy: 0.5506
1423/1563 [==========================>...] - ETA: 0s - loss: 1.2842 - accuracy: 0.5508
1461/1563 [===========================>..] - ETA: 0s - loss: 1.2853 - accuracy: 0.5502
1500/1563 [===========================>..] - ETA: 0s - loss: 1.2856 - accuracy: 0.5501
1539/1563 [============================>.] - ETA: 0s - loss: 1.2850 - accuracy: 0.5504
1563/1563 [==============================] - 2s 1ms/step - loss: 1.2854 - accuracy: 0.5502
Epoch 10/10
1/1563 [..............................] - ETA: 2s - loss: 1.1715 - accuracy: 0.5312
40/1563 [..............................] - ETA: 1s - loss: 1.2416 - accuracy: 0.5547
78/1563 [>.............................] - ETA: 1s - loss: 1.2540 - accuracy: 0.5537
116/1563 [=>............................] - ETA: 1s - loss: 1.2625 - accuracy: 0.5568
154/1563 [=>............................] - ETA: 1s - loss: 1.2630 - accuracy: 0.5584
192/1563 [==>...........................] - ETA: 1s - loss: 1.2602 - accuracy: 0.5560
230/1563 [===>..........................] - ETA: 1s - loss: 1.2597 - accuracy: 0.5565
268/1563 [====>.........................] - ETA: 1s - loss: 1.2597 - accuracy: 0.5594
306/1563 [====>.........................] - ETA: 1s - loss: 1.2602 - accuracy: 0.5612
343/1563 [=====>........................] - ETA: 1s - loss: 1.2662 - accuracy: 0.5599
381/1563 [======>.......................] - ETA: 1s - loss: 1.2641 - accuracy: 0.5606
419/1563 [=======>......................] - ETA: 1s - loss: 1.2635 - accuracy: 0.5600
457/1563 [=======>......................] - ETA: 1s - loss: 1.2603 - accuracy: 0.5599
495/1563 [========>.....................] - ETA: 1s - loss: 1.2561 - accuracy: 0.5626
533/1563 [=========>....................] - ETA: 1s - loss: 1.2559 - accuracy: 0.5630
571/1563 [=========>....................] - ETA: 1s - loss: 1.2492 - accuracy: 0.5649
609/1563 [==========>...................] - ETA: 1s - loss: 1.2513 - accuracy: 0.5635
647/1563 [===========>..................] - ETA: 1s - loss: 1.2508 - accuracy: 0.5637
685/1563 [============>.................] - ETA: 1s - loss: 1.2501 - accuracy: 0.5630
723/1563 [============>.................] - ETA: 1s - loss: 1.2480 - accuracy: 0.5640
761/1563 [=============>................] - ETA: 1s - loss: 1.2517 - accuracy: 0.5628
799/1563 [==============>...............] - ETA: 1s - loss: 1.2525 - accuracy: 0.5623
837/1563 [===============>..............] - ETA: 0s - loss: 1.2504 - accuracy: 0.5627
875/1563 [===============>..............] - ETA: 0s - loss: 1.2510 - accuracy: 0.5622
913/1563 [================>.............] - ETA: 0s - loss: 1.2520 - accuracy: 0.5616
951/1563 [=================>............] - ETA: 0s - loss: 1.2525 - accuracy: 0.5618
989/1563 [=================>............] - ETA: 0s - loss: 1.2531 - accuracy: 0.5620
1027/1563 [==================>...........] - ETA: 0s - loss: 1.2530 - accuracy: 0.5618
1065/1563 [===================>..........] - ETA: 0s - loss: 1.2541 - accuracy: 0.5619
1104/1563 [====================>.........] - ETA: 0s - loss: 1.2538 - accuracy: 0.5616
1142/1563 [====================>.........] - ETA: 0s - loss: 1.2528 - accuracy: 0.5618
1180/1563 [=====================>........] - ETA: 0s - loss: 1.2527 - accuracy: 0.5619
1218/1563 [======================>.......] - ETA: 0s - loss: 1.2516 - accuracy: 0.5620
1256/1563 [=======================>......] - ETA: 0s - loss: 1.2528 - accuracy: 0.5613
1294/1563 [=======================>......] - ETA: 0s - loss: 1.2534 - accuracy: 0.5614
1332/1563 [========================>.....] - ETA: 0s - loss: 1.2538 - accuracy: 0.5613
1370/1563 [=========================>....] - ETA: 0s - loss: 1.2542 - accuracy: 0.5615
1408/1563 [==========================>...] - ETA: 0s - loss: 1.2540 - accuracy: 0.5616
1447/1563 [==========================>...] - ETA: 0s - loss: 1.2536 - accuracy: 0.5616
1485/1563 [===========================>..] - ETA: 0s - loss: 1.2532 - accuracy: 0.5616
1523/1563 [============================>.] - ETA: 0s - loss: 1.2530 - accuracy: 0.5618
1561/1563 [============================>.] - ETA: 0s - loss: 1.2530 - accuracy: 0.5618
1563/1563 [==============================] - 2s 1ms/step - loss: 1.2531 - accuracy: 0.5618
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
flatten (Flatten) (None, 3072) 0
dense (Dense) (None, 3000) 9219000
dense_1 (Dense) (None, 1000) 3001000
dense_2 (Dense) (None, 10) 10010
=================================================================
Total params: 12,230,010
Trainable params: 12,230,010
Non-trainable params: 0
_________________________________________________________________
None
wall time seconds (without loading test data set)= 35.70632362365723
hydro03
[SSH] completed
[SSH] exit-status: 0
[workspace] $ /bin/sh -xe /tmp/jenkins4852163297547659299.sh
+ scp 'HYDRO_REMOTE:~svchydrojenkins/python/tensorflow_time.txt' /var/lib/jenkins/jobs/tensorflow/workspace
Recording plot data
Saving plot series data from: /var/lib/jenkins/jobs/tensorflow/workspace/tensorflow_time.txt
Finished: SUCCESS
![[Jenkins]](/static/755cd4af/images/svgs/logo.svg){kind=logo}
