Started by timer
Running as SYSTEM
Building in workspace /var/lib/jenkins/jobs/tensorflow/workspace
[SSH] script:
module load anaconda3_gpu
module load cudnn
cd python
rm -f tensorflow_test.txt
srun --nodes=1 --ntasks-per-node=1 --mem=16g \
--time=00:05:00 \
--partition=a100rome \
--account=bbmb-hydro \
--gpus-per-node=1 \
--gpu-bind=closest \
python3 cifar10gpu.py | tee tensorflow_test.txt
YVALUE=`grep "wall time seconds" tensorflow_test.txt | cut -d'=' -f2`
echo YVALUE=$YVALUE > tensorflow_time.txt
[SSH] executing...
srun: job 99138 queued and waiting for resources
srun: job 99138 has been allocated resources
2024-12-04 07:43:02.884064: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-12-04 07:43:03.156396: E tensorflow/stream_executor/cuda/cuda_blas.cc:2981] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2024-12-04 07:43:04.379772: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /sw/spack/hydrogpu-2022-06/apps/cudnn/8.4.0.27-11.6-gcc-11.3.0-qbhsfav/lib:/sw/spack/hydrogpu-2022-06/apps/cuda/11.7.0-gcc-11.3.0-3ysno6b/lib64:/sw/spack/hydrogpu-2022-06/apps/openmpi/4.1.4-gcc-11.3.0-i6koipj/lib:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib64:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib
2024-12-04 07:43:04.379853: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /sw/spack/hydrogpu-2022-06/apps/cudnn/8.4.0.27-11.6-gcc-11.3.0-qbhsfav/lib:/sw/spack/hydrogpu-2022-06/apps/cuda/11.7.0-gcc-11.3.0-3ysno6b/lib64:/sw/spack/hydrogpu-2022-06/apps/openmpi/4.1.4-gcc-11.3.0-i6koipj/lib:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib64:/sw/spack/hydrogpu-2022-06/apps/gcc/11.3.0-gcc-8.4.1-kroofxa/lib
2024-12-04 07:43:04.379859: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
2024-12-04 07:43:08.118673: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2024-12-04 07:43:08.490203: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1616] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 79148 MB memory: -> device: 0, name: NVIDIA A100 80GB PCIe, pci bus id: 0000:21:00.0, compute capability: 8.0
(50000, 32, 32, 3) (10000, 32, 32, 3)
Epoch 1/10
2024-12-04 07:43:10.624436: I tensorflow/stream_executor/cuda/cuda_blas.cc:1614] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
1/1563 [..............................] - ETA: 29:34 - loss: 2.4737 - accuracy: 0.0938
37/1563 [..............................] - ETA: 2s - loss: 2.2658 - accuracy: 0.1529
75/1563 [>.............................] - ETA: 2s - loss: 2.1777 - accuracy: 0.1967
113/1563 [=>............................] - ETA: 1s - loss: 2.1234 - accuracy: 0.2163
151/1563 [=>............................] - ETA: 1s - loss: 2.0879 - accuracy: 0.2345
189/1563 [==>...........................] - ETA: 1s - loss: 2.0657 - accuracy: 0.2429
228/1563 [===>..........................] - ETA: 1s - loss: 2.0387 - accuracy: 0.2562
267/1563 [====>.........................] - ETA: 1s - loss: 2.0177 - accuracy: 0.2677
305/1563 [====>.........................] - ETA: 1s - loss: 2.0005 - accuracy: 0.2760
343/1563 [=====>........................] - ETA: 1s - loss: 1.9879 - accuracy: 0.2812
382/1563 [======>.......................] - ETA: 1s - loss: 1.9769 - accuracy: 0.2845
420/1563 [=======>......................] - ETA: 1s - loss: 1.9657 - accuracy: 0.2901
458/1563 [=======>......................] - ETA: 1s - loss: 1.9555 - accuracy: 0.2948
497/1563 [========>.....................] - ETA: 1s - loss: 1.9498 - accuracy: 0.2984
536/1563 [=========>....................] - ETA: 1s - loss: 1.9409 - accuracy: 0.3027
575/1563 [==========>...................] - ETA: 1s - loss: 1.9316 - accuracy: 0.3072
614/1563 [==========>...................] - ETA: 1s - loss: 1.9233 - accuracy: 0.3110
653/1563 [===========>..................] - ETA: 1s - loss: 1.9179 - accuracy: 0.3123
692/1563 [============>.................] - ETA: 1s - loss: 1.9105 - accuracy: 0.3148
731/1563 [=============>................] - ETA: 1s - loss: 1.9033 - accuracy: 0.3177
770/1563 [=============>................] - ETA: 1s - loss: 1.8982 - accuracy: 0.3206
809/1563 [==============>...............] - ETA: 0s - loss: 1.8916 - accuracy: 0.3232
848/1563 [===============>..............] - ETA: 0s - loss: 1.8861 - accuracy: 0.3255
887/1563 [================>.............] - ETA: 0s - loss: 1.8809 - accuracy: 0.3273
926/1563 [================>.............] - ETA: 0s - loss: 1.8756 - accuracy: 0.3303
965/1563 [=================>............] - ETA: 0s - loss: 1.8719 - accuracy: 0.3319
1004/1563 [==================>...........] - ETA: 0s - loss: 1.8670 - accuracy: 0.3340
1043/1563 [===================>..........] - ETA: 0s - loss: 1.8626 - accuracy: 0.3369
1082/1563 [===================>..........] - ETA: 0s - loss: 1.8590 - accuracy: 0.3377
1121/1563 [====================>.........] - ETA: 0s - loss: 1.8554 - accuracy: 0.3388
1160/1563 [=====================>........] - ETA: 0s - loss: 1.8513 - accuracy: 0.3405
1199/1563 [======================>.......] - ETA: 0s - loss: 1.8470 - accuracy: 0.3419
1239/1563 [======================>.......] - ETA: 0s - loss: 1.8428 - accuracy: 0.3440
1278/1563 [=======================>......] - ETA: 0s - loss: 1.8395 - accuracy: 0.3455
1318/1563 [========================>.....] - ETA: 0s - loss: 1.8348 - accuracy: 0.3469
1358/1563 [=========================>....] - ETA: 0s - loss: 1.8311 - accuracy: 0.3481
1397/1563 [=========================>....] - ETA: 0s - loss: 1.8280 - accuracy: 0.3492
1436/1563 [==========================>...] - ETA: 0s - loss: 1.8244 - accuracy: 0.3503
1475/1563 [===========================>..] - ETA: 0s - loss: 1.8205 - accuracy: 0.3520
1514/1563 [============================>.] - ETA: 0s - loss: 1.8177 - accuracy: 0.3532
1553/1563 [============================>.] - ETA: 0s - loss: 1.8152 - accuracy: 0.3538
1563/1563 [==============================] - 3s 1ms/step - loss: 1.8145 - accuracy: 0.3541
Epoch 2/10
1/1563 [..............................] - ETA: 2s - loss: 1.7555 - accuracy: 0.3438
40/1563 [..............................] - ETA: 1s - loss: 1.6935 - accuracy: 0.4031
79/1563 [>.............................] - ETA: 1s - loss: 1.6871 - accuracy: 0.4078
118/1563 [=>............................] - ETA: 1s - loss: 1.6708 - accuracy: 0.4102
157/1563 [==>...........................] - ETA: 1s - loss: 1.6708 - accuracy: 0.4070
196/1563 [==>...........................] - ETA: 1s - loss: 1.6678 - accuracy: 0.4118
235/1563 [===>..........................] - ETA: 1s - loss: 1.6727 - accuracy: 0.4100
274/1563 [====>.........................] - ETA: 1s - loss: 1.6709 - accuracy: 0.4124
313/1563 [=====>........................] - ETA: 1s - loss: 1.6684 - accuracy: 0.4142
352/1563 [=====>........................] - ETA: 1s - loss: 1.6608 - accuracy: 0.4151
391/1563 [======>.......................] - ETA: 1s - loss: 1.6527 - accuracy: 0.4166
430/1563 [=======>......................] - ETA: 1s - loss: 1.6551 - accuracy: 0.4161
469/1563 [========>.....................] - ETA: 1s - loss: 1.6534 - accuracy: 0.4172
506/1563 [========>.....................] - ETA: 1s - loss: 1.6522 - accuracy: 0.4164
545/1563 [=========>....................] - ETA: 1s - loss: 1.6510 - accuracy: 0.4171
584/1563 [==========>...................] - ETA: 1s - loss: 1.6506 - accuracy: 0.4167
623/1563 [==========>...................] - ETA: 1s - loss: 1.6506 - accuracy: 0.4162
662/1563 [===========>..................] - ETA: 1s - loss: 1.6489 - accuracy: 0.4164
701/1563 [============>.................] - ETA: 1s - loss: 1.6493 - accuracy: 0.4165
740/1563 [=============>................] - ETA: 1s - loss: 1.6478 - accuracy: 0.4174
780/1563 [=============>................] - ETA: 1s - loss: 1.6460 - accuracy: 0.4174
818/1563 [==============>...............] - ETA: 0s - loss: 1.6450 - accuracy: 0.4170
857/1563 [===============>..............] - ETA: 0s - loss: 1.6452 - accuracy: 0.4168
896/1563 [================>.............] - ETA: 0s - loss: 1.6461 - accuracy: 0.4171
935/1563 [================>.............] - ETA: 0s - loss: 1.6447 - accuracy: 0.4180
974/1563 [=================>............] - ETA: 0s - loss: 1.6439 - accuracy: 0.4189
1013/1563 [==================>...........] - ETA: 0s - loss: 1.6412 - accuracy: 0.4194
1052/1563 [===================>..........] - ETA: 0s - loss: 1.6404 - accuracy: 0.4202
1091/1563 [===================>..........] - ETA: 0s - loss: 1.6387 - accuracy: 0.4207
1130/1563 [====================>.........] - ETA: 0s - loss: 1.6375 - accuracy: 0.4217
1169/1563 [=====================>........] - ETA: 0s - loss: 1.6357 - accuracy: 0.4220
1208/1563 [======================>.......] - ETA: 0s - loss: 1.6336 - accuracy: 0.4227
1247/1563 [======================>.......] - ETA: 0s - loss: 1.6323 - accuracy: 0.4231
1286/1563 [=======================>......] - ETA: 0s - loss: 1.6331 - accuracy: 0.4233
1325/1563 [========================>.....] - ETA: 0s - loss: 1.6321 - accuracy: 0.4234
1365/1563 [=========================>....] - ETA: 0s - loss: 1.6299 - accuracy: 0.4242
1404/1563 [=========================>....] - ETA: 0s - loss: 1.6287 - accuracy: 0.4243
1440/1563 [==========================>...] - ETA: 0s - loss: 1.6277 - accuracy: 0.4247
1478/1563 [===========================>..] - ETA: 0s - loss: 1.6260 - accuracy: 0.4254
1517/1563 [============================>.] - ETA: 0s - loss: 1.6258 - accuracy: 0.4255
1556/1563 [============================>.] - ETA: 0s - loss: 1.6255 - accuracy: 0.4258
1563/1563 [==============================] - 2s 1ms/step - loss: 1.6257 - accuracy: 0.4258
Epoch 3/10
1/1563 [..............................] - ETA: 2s - loss: 1.7698 - accuracy: 0.3125
40/1563 [..............................] - ETA: 1s - loss: 1.5644 - accuracy: 0.4437
79/1563 [>.............................] - ETA: 1s - loss: 1.5715 - accuracy: 0.4454
118/1563 [=>............................] - ETA: 1s - loss: 1.5626 - accuracy: 0.4499
157/1563 [==>...........................] - ETA: 1s - loss: 1.5675 - accuracy: 0.4486
196/1563 [==>...........................] - ETA: 1s - loss: 1.5624 - accuracy: 0.4482
235/1563 [===>..........................] - ETA: 1s - loss: 1.5627 - accuracy: 0.4468
274/1563 [====>.........................] - ETA: 1s - loss: 1.5600 - accuracy: 0.4500
313/1563 [=====>........................] - ETA: 1s - loss: 1.5558 - accuracy: 0.4508
352/1563 [=====>........................] - ETA: 1s - loss: 1.5541 - accuracy: 0.4525
391/1563 [======>.......................] - ETA: 1s - loss: 1.5530 - accuracy: 0.4524
430/1563 [=======>......................] - ETA: 1s - loss: 1.5529 - accuracy: 0.4505
469/1563 [========>.....................] - ETA: 1s - loss: 1.5578 - accuracy: 0.4480
508/1563 [========>.....................] - ETA: 1s - loss: 1.5592 - accuracy: 0.4482
547/1563 [=========>....................] - ETA: 1s - loss: 1.5571 - accuracy: 0.4484
586/1563 [==========>...................] - ETA: 1s - loss: 1.5566 - accuracy: 0.4479
625/1563 [==========>...................] - ETA: 1s - loss: 1.5561 - accuracy: 0.4490
664/1563 [===========>..................] - ETA: 1s - loss: 1.5577 - accuracy: 0.4482
703/1563 [============>.................] - ETA: 1s - loss: 1.5576 - accuracy: 0.4481
742/1563 [=============>................] - ETA: 1s - loss: 1.5574 - accuracy: 0.4482
781/1563 [=============>................] - ETA: 1s - loss: 1.5565 - accuracy: 0.4487
820/1563 [==============>...............] - ETA: 0s - loss: 1.5578 - accuracy: 0.4494
859/1563 [===============>..............] - ETA: 0s - loss: 1.5573 - accuracy: 0.4494
896/1563 [================>.............] - ETA: 0s - loss: 1.5580 - accuracy: 0.4490
935/1563 [================>.............] - ETA: 0s - loss: 1.5562 - accuracy: 0.4511
974/1563 [=================>............] - ETA: 0s - loss: 1.5562 - accuracy: 0.4507
1013/1563 [==================>...........] - ETA: 0s - loss: 1.5556 - accuracy: 0.4513
1052/1563 [===================>..........] - ETA: 0s - loss: 1.5546 - accuracy: 0.4516
1091/1563 [===================>..........] - ETA: 0s - loss: 1.5532 - accuracy: 0.4513
1130/1563 [====================>.........] - ETA: 0s - loss: 1.5525 - accuracy: 0.4522
1169/1563 [=====================>........] - ETA: 0s - loss: 1.5521 - accuracy: 0.4518
1208/1563 [======================>.......] - ETA: 0s - loss: 1.5510 - accuracy: 0.4526
1247/1563 [======================>.......] - ETA: 0s - loss: 1.5519 - accuracy: 0.4523
1286/1563 [=======================>......] - ETA: 0s - loss: 1.5512 - accuracy: 0.4530
1324/1563 [========================>.....] - ETA: 0s - loss: 1.5503 - accuracy: 0.4533
1363/1563 [=========================>....] - ETA: 0s - loss: 1.5485 - accuracy: 0.4541
1402/1563 [=========================>....] - ETA: 0s - loss: 1.5479 - accuracy: 0.4546
1441/1563 [==========================>...] - ETA: 0s - loss: 1.5489 - accuracy: 0.4544
1480/1563 [===========================>..] - ETA: 0s - loss: 1.5485 - accuracy: 0.4548
1519/1563 [============================>.] - ETA: 0s - loss: 1.5474 - accuracy: 0.4550
1558/1563 [============================>.] - ETA: 0s - loss: 1.5455 - accuracy: 0.4557
1563/1563 [==============================] - 2s 1ms/step - loss: 1.5454 - accuracy: 0.4558
Epoch 4/10
1/1563 [..............................] - ETA: 2s - loss: 1.7813 - accuracy: 0.4375
39/1563 [..............................] - ETA: 2s - loss: 1.5356 - accuracy: 0.4591
77/1563 [>.............................] - ETA: 1s - loss: 1.5044 - accuracy: 0.4740
116/1563 [=>............................] - ETA: 1s - loss: 1.5019 - accuracy: 0.4714
155/1563 [=>............................] - ETA: 1s - loss: 1.5142 - accuracy: 0.4702
194/1563 [==>...........................] - ETA: 1s - loss: 1.5182 - accuracy: 0.4670
233/1563 [===>..........................] - ETA: 1s - loss: 1.5161 - accuracy: 0.4666
272/1563 [====>.........................] - ETA: 1s - loss: 1.5040 - accuracy: 0.4713
309/1563 [====>.........................] - ETA: 1s - loss: 1.5026 - accuracy: 0.4714
348/1563 [=====>........................] - ETA: 1s - loss: 1.5064 - accuracy: 0.4681
387/1563 [======>.......................] - ETA: 1s - loss: 1.5007 - accuracy: 0.4675
426/1563 [=======>......................] - ETA: 1s - loss: 1.5000 - accuracy: 0.4676
465/1563 [=======>......................] - ETA: 1s - loss: 1.4970 - accuracy: 0.4685
504/1563 [========>.....................] - ETA: 1s - loss: 1.4951 - accuracy: 0.4701
543/1563 [=========>....................] - ETA: 1s - loss: 1.4952 - accuracy: 0.4704
582/1563 [==========>...................] - ETA: 1s - loss: 1.4960 - accuracy: 0.4711
621/1563 [==========>...................] - ETA: 1s - loss: 1.4963 - accuracy: 0.4707
660/1563 [===========>..................] - ETA: 1s - loss: 1.4969 - accuracy: 0.4708
699/1563 [============>.................] - ETA: 1s - loss: 1.4992 - accuracy: 0.4709
738/1563 [=============>................] - ETA: 1s - loss: 1.5011 - accuracy: 0.4701
777/1563 [=============>................] - ETA: 1s - loss: 1.5005 - accuracy: 0.4697
816/1563 [==============>...............] - ETA: 0s - loss: 1.4978 - accuracy: 0.4702
855/1563 [===============>..............] - ETA: 0s - loss: 1.4964 - accuracy: 0.4706
893/1563 [================>.............] - ETA: 0s - loss: 1.4946 - accuracy: 0.4716
932/1563 [================>.............] - ETA: 0s - loss: 1.4943 - accuracy: 0.4719
970/1563 [=================>............] - ETA: 0s - loss: 1.4926 - accuracy: 0.4723
1009/1563 [==================>...........] - ETA: 0s - loss: 1.4906 - accuracy: 0.4727
1048/1563 [===================>..........] - ETA: 0s - loss: 1.4900 - accuracy: 0.4734
1088/1563 [===================>..........] - ETA: 0s - loss: 1.4887 - accuracy: 0.4739
1128/1563 [====================>.........] - ETA: 0s - loss: 1.4892 - accuracy: 0.4737
1167/1563 [=====================>........] - ETA: 0s - loss: 1.4885 - accuracy: 0.4738
1206/1563 [======================>.......] - ETA: 0s - loss: 1.4875 - accuracy: 0.4738
1245/1563 [======================>.......] - ETA: 0s - loss: 1.4865 - accuracy: 0.4741
1284/1563 [=======================>......] - ETA: 0s - loss: 1.4863 - accuracy: 0.4743
1323/1563 [========================>.....] - ETA: 0s - loss: 1.4856 - accuracy: 0.4745
1362/1563 [=========================>....] - ETA: 0s - loss: 1.4846 - accuracy: 0.4750
1401/1563 [=========================>....] - ETA: 0s - loss: 1.4841 - accuracy: 0.4760
1440/1563 [==========================>...] - ETA: 0s - loss: 1.4833 - accuracy: 0.4764
1479/1563 [===========================>..] - ETA: 0s - loss: 1.4841 - accuracy: 0.4760
1517/1563 [============================>.] - ETA: 0s - loss: 1.4837 - accuracy: 0.4763
1556/1563 [============================>.] - ETA: 0s - loss: 1.4846 - accuracy: 0.4765
1563/1563 [==============================] - 2s 1ms/step - loss: 1.4842 - accuracy: 0.4768
Epoch 5/10
1/1563 [..............................] - ETA: 1s - loss: 1.5545 - accuracy: 0.4375
40/1563 [..............................] - ETA: 1s - loss: 1.4913 - accuracy: 0.4852
79/1563 [>.............................] - ETA: 1s - loss: 1.4595 - accuracy: 0.4869
118/1563 [=>............................] - ETA: 1s - loss: 1.4605 - accuracy: 0.4886
157/1563 [==>...........................] - ETA: 1s - loss: 1.4608 - accuracy: 0.4865
196/1563 [==>...........................] - ETA: 1s - loss: 1.4452 - accuracy: 0.4903
235/1563 [===>..........................] - ETA: 1s - loss: 1.4509 - accuracy: 0.4875
274/1563 [====>.........................] - ETA: 1s - loss: 1.4520 - accuracy: 0.4877
313/1563 [=====>........................] - ETA: 1s - loss: 1.4525 - accuracy: 0.4901
352/1563 [=====>........................] - ETA: 1s - loss: 1.4506 - accuracy: 0.4903
391/1563 [======>.......................] - ETA: 1s - loss: 1.4482 - accuracy: 0.4915
430/1563 [=======>......................] - ETA: 1s - loss: 1.4459 - accuracy: 0.4928
469/1563 [========>.....................] - ETA: 1s - loss: 1.4409 - accuracy: 0.4936
508/1563 [========>.....................] - ETA: 1s - loss: 1.4409 - accuracy: 0.4938
546/1563 [=========>....................] - ETA: 1s - loss: 1.4389 - accuracy: 0.4928
585/1563 [==========>...................] - ETA: 1s - loss: 1.4385 - accuracy: 0.4938
624/1563 [==========>...................] - ETA: 1s - loss: 1.4406 - accuracy: 0.4935
663/1563 [===========>..................] - ETA: 1s - loss: 1.4404 - accuracy: 0.4941
702/1563 [============>.................] - ETA: 1s - loss: 1.4399 - accuracy: 0.4935
741/1563 [=============>................] - ETA: 1s - loss: 1.4380 - accuracy: 0.4946
780/1563 [=============>................] - ETA: 1s - loss: 1.4375 - accuracy: 0.4944
819/1563 [==============>...............] - ETA: 0s - loss: 1.4390 - accuracy: 0.4931
858/1563 [===============>..............] - ETA: 0s - loss: 1.4399 - accuracy: 0.4934
897/1563 [================>.............] - ETA: 0s - loss: 1.4383 - accuracy: 0.4941
936/1563 [================>.............] - ETA: 0s - loss: 1.4376 - accuracy: 0.4940
975/1563 [=================>............] - ETA: 0s - loss: 1.4370 - accuracy: 0.4943
1014/1563 [==================>...........] - ETA: 0s - loss: 1.4367 - accuracy: 0.4946
1053/1563 [===================>..........] - ETA: 0s - loss: 1.4370 - accuracy: 0.4939
1092/1563 [===================>..........] - ETA: 0s - loss: 1.4376 - accuracy: 0.4938
1131/1563 [====================>.........] - ETA: 0s - loss: 1.4374 - accuracy: 0.4933
1170/1563 [=====================>........] - ETA: 0s - loss: 1.4383 - accuracy: 0.4938
1209/1563 [======================>.......] - ETA: 0s - loss: 1.4378 - accuracy: 0.4940
1248/1563 [======================>.......] - ETA: 0s - loss: 1.4366 - accuracy: 0.4947
1287/1563 [=======================>......] - ETA: 0s - loss: 1.4353 - accuracy: 0.4950
1326/1563 [========================>.....] - ETA: 0s - loss: 1.4344 - accuracy: 0.4951
1365/1563 [=========================>....] - ETA: 0s - loss: 1.4338 - accuracy: 0.4948
1404/1563 [=========================>....] - ETA: 0s - loss: 1.4329 - accuracy: 0.4951
1443/1563 [==========================>...] - ETA: 0s - loss: 1.4326 - accuracy: 0.4954
1482/1563 [===========================>..] - ETA: 0s - loss: 1.4326 - accuracy: 0.4952
1520/1563 [============================>.] - ETA: 0s - loss: 1.4338 - accuracy: 0.4949
1559/1563 [============================>.] - ETA: 0s - loss: 1.4328 - accuracy: 0.4947
1563/1563 [==============================] - 2s 1ms/step - loss: 1.4334 - accuracy: 0.4945
Epoch 6/10
1/1563 [..............................] - ETA: 2s - loss: 1.3085 - accuracy: 0.5312
40/1563 [..............................] - ETA: 1s - loss: 1.4221 - accuracy: 0.5086
78/1563 [>.............................] - ETA: 1s - loss: 1.3806 - accuracy: 0.5240
117/1563 [=>............................] - ETA: 1s - loss: 1.3885 - accuracy: 0.5128
156/1563 [=>............................] - ETA: 1s - loss: 1.3839 - accuracy: 0.5170
195/1563 [==>...........................] - ETA: 1s - loss: 1.3842 - accuracy: 0.5163
234/1563 [===>..........................] - ETA: 1s - loss: 1.3934 - accuracy: 0.5108
273/1563 [====>.........................] - ETA: 1s - loss: 1.3935 - accuracy: 0.5100
312/1563 [====>.........................] - ETA: 1s - loss: 1.3977 - accuracy: 0.5089
351/1563 [=====>........................] - ETA: 1s - loss: 1.3969 - accuracy: 0.5116
390/1563 [======>.......................] - ETA: 1s - loss: 1.3996 - accuracy: 0.5095
429/1563 [=======>......................] - ETA: 1s - loss: 1.3983 - accuracy: 0.5098
468/1563 [=======>......................] - ETA: 1s - loss: 1.3948 - accuracy: 0.5108
507/1563 [========>.....................] - ETA: 1s - loss: 1.3951 - accuracy: 0.5105
546/1563 [=========>....................] - ETA: 1s - loss: 1.3927 - accuracy: 0.5118
585/1563 [==========>...................] - ETA: 1s - loss: 1.3921 - accuracy: 0.5129
624/1563 [==========>...................] - ETA: 1s - loss: 1.3946 - accuracy: 0.5118
663/1563 [===========>..................] - ETA: 1s - loss: 1.3980 - accuracy: 0.5107
702/1563 [============>.................] - ETA: 1s - loss: 1.3981 - accuracy: 0.5106
740/1563 [=============>................] - ETA: 1s - loss: 1.3960 - accuracy: 0.5117
779/1563 [=============>................] - ETA: 1s - loss: 1.3964 - accuracy: 0.5117
818/1563 [==============>...............] - ETA: 0s - loss: 1.3957 - accuracy: 0.5113
857/1563 [===============>..............] - ETA: 0s - loss: 1.3967 - accuracy: 0.5103
895/1563 [================>.............] - ETA: 0s - loss: 1.3963 - accuracy: 0.5099
934/1563 [================>.............] - ETA: 0s - loss: 1.3954 - accuracy: 0.5101
972/1563 [=================>............] - ETA: 0s - loss: 1.3937 - accuracy: 0.5112
1010/1563 [==================>...........] - ETA: 0s - loss: 1.3926 - accuracy: 0.5110
1049/1563 [===================>..........] - ETA: 0s - loss: 1.3926 - accuracy: 0.5117
1088/1563 [===================>..........] - ETA: 0s - loss: 1.3925 - accuracy: 0.5109
1127/1563 [====================>.........] - ETA: 0s - loss: 1.3931 - accuracy: 0.5109
1166/1563 [=====================>........] - ETA: 0s - loss: 1.3933 - accuracy: 0.5107
1205/1563 [======================>.......] - ETA: 0s - loss: 1.3945 - accuracy: 0.5101
1244/1563 [======================>.......] - ETA: 0s - loss: 1.3939 - accuracy: 0.5100
1283/1563 [=======================>......] - ETA: 0s - loss: 1.3924 - accuracy: 0.5105
1322/1563 [========================>.....] - ETA: 0s - loss: 1.3918 - accuracy: 0.5105
1361/1563 [=========================>....] - ETA: 0s - loss: 1.3920 - accuracy: 0.5105
1400/1563 [=========================>....] - ETA: 0s - loss: 1.3911 - accuracy: 0.5111
1439/1563 [==========================>...] - ETA: 0s - loss: 1.3913 - accuracy: 0.5106
1478/1563 [===========================>..] - ETA: 0s - loss: 1.3920 - accuracy: 0.5106
1517/1563 [============================>.] - ETA: 0s - loss: 1.3917 - accuracy: 0.5109
1556/1563 [============================>.] - ETA: 0s - loss: 1.3911 - accuracy: 0.5113
1563/1563 [==============================] - 2s 1ms/step - loss: 1.3909 - accuracy: 0.5113
Epoch 7/10
1/1563 [..............................] - ETA: 2s - loss: 1.4575 - accuracy: 0.4062
40/1563 [..............................] - ETA: 1s - loss: 1.3506 - accuracy: 0.5188
79/1563 [>.............................] - ETA: 1s - loss: 1.3423 - accuracy: 0.5146
118/1563 [=>............................] - ETA: 1s - loss: 1.3504 - accuracy: 0.5169
157/1563 [==>...........................] - ETA: 1s - loss: 1.3410 - accuracy: 0.5229
196/1563 [==>...........................] - ETA: 1s - loss: 1.3480 - accuracy: 0.5236
235/1563 [===>..........................] - ETA: 1s - loss: 1.3522 - accuracy: 0.5250
274/1563 [====>.........................] - ETA: 1s - loss: 1.3482 - accuracy: 0.5263
313/1563 [=====>........................] - ETA: 1s - loss: 1.3501 - accuracy: 0.5260
352/1563 [=====>........................] - ETA: 1s - loss: 1.3530 - accuracy: 0.5259
391/1563 [======>.......................] - ETA: 1s - loss: 1.3483 - accuracy: 0.5272
430/1563 [=======>......................] - ETA: 1s - loss: 1.3490 - accuracy: 0.5267
469/1563 [========>.....................] - ETA: 1s - loss: 1.3478 - accuracy: 0.5277
507/1563 [========>.....................] - ETA: 1s - loss: 1.3504 - accuracy: 0.5265
546/1563 [=========>....................] - ETA: 1s - loss: 1.3494 - accuracy: 0.5274
585/1563 [==========>...................] - ETA: 1s - loss: 1.3512 - accuracy: 0.5262
624/1563 [==========>...................] - ETA: 1s - loss: 1.3539 - accuracy: 0.5256
663/1563 [===========>..................] - ETA: 1s - loss: 1.3558 - accuracy: 0.5256
702/1563 [============>.................] - ETA: 1s - loss: 1.3536 - accuracy: 0.5268
741/1563 [=============>................] - ETA: 1s - loss: 1.3525 - accuracy: 0.5265
781/1563 [=============>................] - ETA: 1s - loss: 1.3549 - accuracy: 0.5253
820/1563 [==============>...............] - ETA: 0s - loss: 1.3536 - accuracy: 0.5260
859/1563 [===============>..............] - ETA: 0s - loss: 1.3522 - accuracy: 0.5258
898/1563 [================>.............] - ETA: 0s - loss: 1.3514 - accuracy: 0.5259
937/1563 [================>.............] - ETA: 0s - loss: 1.3533 - accuracy: 0.5247
976/1563 [=================>............] - ETA: 0s - loss: 1.3521 - accuracy: 0.5248
1015/1563 [==================>...........] - ETA: 0s - loss: 1.3538 - accuracy: 0.5236
1054/1563 [===================>..........] - ETA: 0s - loss: 1.3539 - accuracy: 0.5236
1093/1563 [===================>..........] - ETA: 0s - loss: 1.3536 - accuracy: 0.5231
1132/1563 [====================>.........] - ETA: 0s - loss: 1.3534 - accuracy: 0.5234
1171/1563 [=====================>........] - ETA: 0s - loss: 1.3535 - accuracy: 0.5241
1209/1563 [======================>.......] - ETA: 0s - loss: 1.3540 - accuracy: 0.5237
1248/1563 [======================>.......] - ETA: 0s - loss: 1.3537 - accuracy: 0.5235
1287/1563 [=======================>......] - ETA: 0s - loss: 1.3531 - accuracy: 0.5240
1326/1563 [========================>.....] - ETA: 0s - loss: 1.3543 - accuracy: 0.5238
1365/1563 [=========================>....] - ETA: 0s - loss: 1.3530 - accuracy: 0.5245
1404/1563 [=========================>....] - ETA: 0s - loss: 1.3528 - accuracy: 0.5248
1443/1563 [==========================>...] - ETA: 0s - loss: 1.3520 - accuracy: 0.5248
1482/1563 [===========================>..] - ETA: 0s - loss: 1.3520 - accuracy: 0.5253
1521/1563 [============================>.] - ETA: 0s - loss: 1.3525 - accuracy: 0.5252
1560/1563 [============================>.] - ETA: 0s - loss: 1.3533 - accuracy: 0.5252
1563/1563 [==============================] - 2s 1ms/step - loss: 1.3533 - accuracy: 0.5252
Epoch 8/10
1/1563 [..............................] - ETA: 1s - loss: 1.4003 - accuracy: 0.5312
40/1563 [..............................] - ETA: 1s - loss: 1.2896 - accuracy: 0.5367
79/1563 [>.............................] - ETA: 1s - loss: 1.2982 - accuracy: 0.5415
117/1563 [=>............................] - ETA: 1s - loss: 1.3091 - accuracy: 0.5339
156/1563 [=>............................] - ETA: 1s - loss: 1.3178 - accuracy: 0.5375
195/1563 [==>...........................] - ETA: 1s - loss: 1.3143 - accuracy: 0.5402
234/1563 [===>..........................] - ETA: 1s - loss: 1.3242 - accuracy: 0.5386
273/1563 [====>.........................] - ETA: 1s - loss: 1.3211 - accuracy: 0.5412
311/1563 [====>.........................] - ETA: 1s - loss: 1.3188 - accuracy: 0.5393
350/1563 [=====>........................] - ETA: 1s - loss: 1.3134 - accuracy: 0.5411
389/1563 [======>.......................] - ETA: 1s - loss: 1.3114 - accuracy: 0.5421
428/1563 [=======>......................] - ETA: 1s - loss: 1.3148 - accuracy: 0.5422
467/1563 [=======>......................] - ETA: 1s - loss: 1.3122 - accuracy: 0.5424
506/1563 [========>.....................] - ETA: 1s - loss: 1.3133 - accuracy: 0.5404
545/1563 [=========>....................] - ETA: 1s - loss: 1.3122 - accuracy: 0.5407
583/1563 [==========>...................] - ETA: 1s - loss: 1.3093 - accuracy: 0.5414
622/1563 [==========>...................] - ETA: 1s - loss: 1.3095 - accuracy: 0.5414
661/1563 [===========>..................] - ETA: 1s - loss: 1.3087 - accuracy: 0.5419
700/1563 [============>.................] - ETA: 1s - loss: 1.3081 - accuracy: 0.5422
739/1563 [=============>................] - ETA: 1s - loss: 1.3088 - accuracy: 0.5410
778/1563 [=============>................] - ETA: 1s - loss: 1.3115 - accuracy: 0.5396
817/1563 [==============>...............] - ETA: 0s - loss: 1.3111 - accuracy: 0.5394
856/1563 [===============>..............] - ETA: 0s - loss: 1.3097 - accuracy: 0.5398
894/1563 [================>.............] - ETA: 0s - loss: 1.3095 - accuracy: 0.5403
933/1563 [================>.............] - ETA: 0s - loss: 1.3114 - accuracy: 0.5392
972/1563 [=================>............] - ETA: 0s - loss: 1.3105 - accuracy: 0.5394
1010/1563 [==================>...........] - ETA: 0s - loss: 1.3107 - accuracy: 0.5390
1049/1563 [===================>..........] - ETA: 0s - loss: 1.3121 - accuracy: 0.5386
1088/1563 [===================>..........] - ETA: 0s - loss: 1.3119 - accuracy: 0.5387
1127/1563 [====================>.........] - ETA: 0s - loss: 1.3127 - accuracy: 0.5385
1166/1563 [=====================>........] - ETA: 0s - loss: 1.3144 - accuracy: 0.5371
1205/1563 [======================>.......] - ETA: 0s - loss: 1.3155 - accuracy: 0.5370
1244/1563 [======================>.......] - ETA: 0s - loss: 1.3137 - accuracy: 0.5372
1283/1563 [=======================>......] - ETA: 0s - loss: 1.3149 - accuracy: 0.5372
1322/1563 [========================>.....] - ETA: 0s - loss: 1.3144 - accuracy: 0.5379
1361/1563 [=========================>....] - ETA: 0s - loss: 1.3153 - accuracy: 0.5377
1400/1563 [=========================>....] - ETA: 0s - loss: 1.3168 - accuracy: 0.5371
1439/1563 [==========================>...] - ETA: 0s - loss: 1.3168 - accuracy: 0.5373
1478/1563 [===========================>..] - ETA: 0s - loss: 1.3178 - accuracy: 0.5370
1516/1563 [============================>.] - ETA: 0s - loss: 1.3180 - accuracy: 0.5371
1555/1563 [============================>.] - ETA: 0s - loss: 1.3180 - accuracy: 0.5369
1563/1563 [==============================] - 2s 1ms/step - loss: 1.3178 - accuracy: 0.5369
Epoch 9/10
1/1563 [..............................] - ETA: 2s - loss: 1.2628 - accuracy: 0.5000
40/1563 [..............................] - ETA: 1s - loss: 1.2575 - accuracy: 0.5672
79/1563 [>.............................] - ETA: 1s - loss: 1.2633 - accuracy: 0.5554
118/1563 [=>............................] - ETA: 1s - loss: 1.2648 - accuracy: 0.5535
157/1563 [==>...........................] - ETA: 1s - loss: 1.2663 - accuracy: 0.5523
196/1563 [==>...........................] - ETA: 1s - loss: 1.2714 - accuracy: 0.5480
234/1563 [===>..........................] - ETA: 1s - loss: 1.2732 - accuracy: 0.5519
273/1563 [====>.........................] - ETA: 1s - loss: 1.2715 - accuracy: 0.5548
312/1563 [====>.........................] - ETA: 1s - loss: 1.2762 - accuracy: 0.5522
351/1563 [=====>........................] - ETA: 1s - loss: 1.2786 - accuracy: 0.5519
390/1563 [======>.......................] - ETA: 1s - loss: 1.2824 - accuracy: 0.5507
429/1563 [=======>......................] - ETA: 1s - loss: 1.2782 - accuracy: 0.5527
468/1563 [=======>......................] - ETA: 1s - loss: 1.2824 - accuracy: 0.5525
507/1563 [========>.....................] - ETA: 1s - loss: 1.2808 - accuracy: 0.5529
546/1563 [=========>....................] - ETA: 1s - loss: 1.2818 - accuracy: 0.5523
585/1563 [==========>...................] - ETA: 1s - loss: 1.2806 - accuracy: 0.5516
624/1563 [==========>...................] - ETA: 1s - loss: 1.2820 - accuracy: 0.5501
663/1563 [===========>..................] - ETA: 1s - loss: 1.2811 - accuracy: 0.5512
702/1563 [============>.................] - ETA: 1s - loss: 1.2805 - accuracy: 0.5513
740/1563 [=============>................] - ETA: 1s - loss: 1.2816 - accuracy: 0.5506
779/1563 [=============>................] - ETA: 1s - loss: 1.2825 - accuracy: 0.5501
818/1563 [==============>...............] - ETA: 0s - loss: 1.2849 - accuracy: 0.5496
857/1563 [===============>..............] - ETA: 0s - loss: 1.2861 - accuracy: 0.5489
896/1563 [================>.............] - ETA: 0s - loss: 1.2847 - accuracy: 0.5498
935/1563 [================>.............] - ETA: 0s - loss: 1.2858 - accuracy: 0.5490
974/1563 [=================>............] - ETA: 0s - loss: 1.2885 - accuracy: 0.5480
1013/1563 [==================>...........] - ETA: 0s - loss: 1.2893 - accuracy: 0.5477
1052/1563 [===================>..........] - ETA: 0s - loss: 1.2885 - accuracy: 0.5484
1091/1563 [===================>..........] - ETA: 0s - loss: 1.2871 - accuracy: 0.5493
1130/1563 [====================>.........] - ETA: 0s - loss: 1.2885 - accuracy: 0.5487
1169/1563 [=====================>........] - ETA: 0s - loss: 1.2872 - accuracy: 0.5491
1208/1563 [======================>.......] - ETA: 0s - loss: 1.2862 - accuracy: 0.5490
1247/1563 [======================>.......] - ETA: 0s - loss: 1.2866 - accuracy: 0.5490
1286/1563 [=======================>......] - ETA: 0s - loss: 1.2863 - accuracy: 0.5493
1325/1563 [========================>.....] - ETA: 0s - loss: 1.2858 - accuracy: 0.5493
1363/1563 [=========================>....] - ETA: 0s - loss: 1.2865 - accuracy: 0.5489
1401/1563 [=========================>....] - ETA: 0s - loss: 1.2864 - accuracy: 0.5491
1440/1563 [==========================>...] - ETA: 0s - loss: 1.2877 - accuracy: 0.5482
1479/1563 [===========================>..] - ETA: 0s - loss: 1.2860 - accuracy: 0.5486
1518/1563 [============================>.] - ETA: 0s - loss: 1.2858 - accuracy: 0.5491
1557/1563 [============================>.] - ETA: 0s - loss: 1.2853 - accuracy: 0.5491
1563/1563 [==============================] - 2s 1ms/step - loss: 1.2851 - accuracy: 0.5492
Epoch 10/10
1/1563 [..............................] - ETA: 2s - loss: 1.0686 - accuracy: 0.7188
40/1563 [..............................] - ETA: 1s - loss: 1.2914 - accuracy: 0.5344
79/1563 [>.............................] - ETA: 1s - loss: 1.2619 - accuracy: 0.5510
118/1563 [=>............................] - ETA: 1s - loss: 1.2468 - accuracy: 0.5617
157/1563 [==>...........................] - ETA: 1s - loss: 1.2558 - accuracy: 0.5541
196/1563 [==>...........................] - ETA: 1s - loss: 1.2544 - accuracy: 0.5542
235/1563 [===>..........................] - ETA: 1s - loss: 1.2554 - accuracy: 0.5513
274/1563 [====>.........................] - ETA: 1s - loss: 1.2549 - accuracy: 0.5520
313/1563 [=====>........................] - ETA: 1s - loss: 1.2608 - accuracy: 0.5521
352/1563 [=====>........................] - ETA: 1s - loss: 1.2661 - accuracy: 0.5510
391/1563 [======>.......................] - ETA: 1s - loss: 1.2651 - accuracy: 0.5538
430/1563 [=======>......................] - ETA: 1s - loss: 1.2658 - accuracy: 0.5549
469/1563 [========>.....................] - ETA: 1s - loss: 1.2611 - accuracy: 0.5575
508/1563 [========>.....................] - ETA: 1s - loss: 1.2618 - accuracy: 0.5579
547/1563 [=========>....................] - ETA: 1s - loss: 1.2609 - accuracy: 0.5580
586/1563 [==========>...................] - ETA: 1s - loss: 1.2611 - accuracy: 0.5581
625/1563 [==========>...................] - ETA: 1s - loss: 1.2624 - accuracy: 0.5570
664/1563 [===========>..................] - ETA: 1s - loss: 1.2648 - accuracy: 0.5556
703/1563 [============>.................] - ETA: 1s - loss: 1.2614 - accuracy: 0.5579
742/1563 [=============>................] - ETA: 1s - loss: 1.2584 - accuracy: 0.5593
781/1563 [=============>................] - ETA: 1s - loss: 1.2610 - accuracy: 0.5580
820/1563 [==============>...............] - ETA: 0s - loss: 1.2635 - accuracy: 0.5566
859/1563 [===============>..............] - ETA: 0s - loss: 1.2632 - accuracy: 0.5561
898/1563 [================>.............] - ETA: 0s - loss: 1.2628 - accuracy: 0.5569
937/1563 [================>.............] - ETA: 0s - loss: 1.2608 - accuracy: 0.5575
976/1563 [=================>............] - ETA: 0s - loss: 1.2582 - accuracy: 0.5582
1015/1563 [==================>...........] - ETA: 0s - loss: 1.2583 - accuracy: 0.5579
1054/1563 [===================>..........] - ETA: 0s - loss: 1.2572 - accuracy: 0.5579
1093/1563 [===================>..........] - ETA: 0s - loss: 1.2557 - accuracy: 0.5584
1130/1563 [====================>.........] - ETA: 0s - loss: 1.2556 - accuracy: 0.5582
1168/1563 [=====================>........] - ETA: 0s - loss: 1.2564 - accuracy: 0.5578
1206/1563 [======================>.......] - ETA: 0s - loss: 1.2576 - accuracy: 0.5573
1245/1563 [======================>.......] - ETA: 0s - loss: 1.2571 - accuracy: 0.5576
1283/1563 [=======================>......] - ETA: 0s - loss: 1.2566 - accuracy: 0.5580
1321/1563 [========================>.....] - ETA: 0s - loss: 1.2565 - accuracy: 0.5580
1360/1563 [=========================>....] - ETA: 0s - loss: 1.2571 - accuracy: 0.5580
1399/1563 [=========================>....] - ETA: 0s - loss: 1.2568 - accuracy: 0.5581
1438/1563 [==========================>...] - ETA: 0s - loss: 1.2564 - accuracy: 0.5580
1477/1563 [===========================>..] - ETA: 0s - loss: 1.2561 - accuracy: 0.5581
1516/1563 [============================>.] - ETA: 0s - loss: 1.2563 - accuracy: 0.5579
1555/1563 [============================>.] - ETA: 0s - loss: 1.2556 - accuracy: 0.5583
1563/1563 [==============================] - 2s 1ms/step - loss: 1.2553 - accuracy: 0.5583
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
flatten (Flatten) (None, 3072) 0
dense (Dense) (None, 3000) 9219000
dense_1 (Dense) (None, 1000) 3001000
dense_2 (Dense) (None, 10) 10010
=================================================================
Total params: 12,230,010
Trainable params: 12,230,010
Non-trainable params: 0
_________________________________________________________________
None
wall time seconds (without loading test data set)= 23.328354597091675
hydro04
[SSH] completed
[SSH] exit-status: 0
[workspace] $ /bin/sh -xe /tmp/jenkins15254801131465170883.sh
+ scp 'HYDRO_REMOTE:~svchydrojenkins/python/tensorflow_time.txt' /var/lib/jenkins/jobs/tensorflow/workspace
Recording plot data
Saving plot series data from: /var/lib/jenkins/jobs/tensorflow/workspace/tensorflow_time.txt
Finished: SUCCESS
![[Jenkins]](/static/755cd4af/images/svgs/logo.svg){kind=logo}
