# # Author: Sebastian Enger, M.Sc. # Date: 4/2/2016 # Website: www.OneTIPP.com # Email: Sebastian.Enger@gmail.com # Topic: Server Manager is creating training tasks in on https://tech.onetipp.net/task.php for Client Slaves # Version: 0.0.1 # 2016-04-09 12:34:31 https://tech.onetipp.net/train/ClientManager.pl.bz2 /var/tmp/onetipp/ apt-get update -y;apt-get upgrade -y pip install -U Chainer pip3 install -U Chainer https://tech.onetipp.net/train/train.py.bz2 /var/tmp/onetipp/ 9f64b87612861709457151d86930f95f https://tech.onetipp.net/train/sample_cpu.py.bz2 /var/tmp/onetipp/ d0b332ff2fea853aebc063275dadf81b https://tech.onetipp.net/train/sample_gpu.py.bz2 /var/tmp/onetipp/ e5c9fc9b21f9f5fa32cfec955e2c32b3 https://tech.onetipp.net/train/CharRNN.py.bz2 /var/tmp/onetipp/ 5d87b4f1816888f37d840d0818408fea IF ClientManager GETS KILL OR HUP COMMAND: -> Send latest "sendStatus" before going down 1. Check if all Perl modules are installed 2. download and install all needed perl modules 3. GET https://tech.onetipp.net/task.php 4. do: ForceInternalDockerImageAPTUpdate 5. do: ForceDeepLearningTrainingSoftwareUpdate 6. do: ForceClientManagerUpdate 6.1: if ForceClientManagerUpdate == 1: ELSE GOTO "STEP 7" 6.2: Calculate MD5 Hash of ClientManager.pl 6.3: if CalcLocalMD5(ClientManager.pl) NOT EQUAL ForceClientManagerUpdate->XML(ClientManager.pl->md5()) 6.4. Download new ClientManager.pl 6.5. Unpack PBZIP2 new ClientManager.pl 6.6. RESTART: ClientManager.pl 7. Create Paths: 7.1. /var/tmp/onetipp 7.2. /var/tmp/onetipp/chainer/ 7.3. Copy ForceDeepLearningTrainingSoftwareUpdate to /var/tmp/onetipp/chainer/ 7.4. Create /var/tmp/onetipp/chainer/data/ 7.5. Create /var/tmp/onetipp/chainer/cv/ 7.5. Create /var/tmp/onetipp/chainer/cv/{$HASHID}/ 7.6. Create /var/tmp/onetipp/chainer/data/{$HASHID}/input.txt 7.7. Start train.py --folder=data/{$HASHID}.txt --cv=cv/{$HASHID} 7.7.1: Keep Process ID of train.py 7.7.2: Keep {XML}->{$HASHID} 7.7.3: Keep Process_STARTTIME in unixtimestamp 7.7.4: Keep {XML}->{LANGUAGE} 7.7.5: Keep {XML}->{worktime} 7.7.6: Sleep: 60 Seconds 8. Wake up from 60 Seconds sleep 8.1: Check if python "train.py" PID is still running 8.2: Check if (Process_STARTTIME*{XML}->{worktime}) >= current_unixtimestamp() 8.2.1: if (Process_STARTTIME*10minuten) >= current_unixtimestamp() -> Send "sendStatus" 8.2.2: if no: sleep another 60 seconds 8.2.3: if yes: -> Send "sendStatus" && kill "train.py" 8.2.3.1: "START AGAIN AT DO STEP 3": GET https://tech.onetipp.net/task.php