2018/03/22

Async Await

ndoe.js 預設的功能都是非同步的,一般來說程式語言都是依序從上執行到下,而 node.js 的話不是,我們來看一下一個簡單的範例

const runTimeout = () => {
    setTimeout(() => {
        console.log('time is up');
    }, 2000);
};

runTimeout();
console.log('done');

以往我們的想像會是先拿到 time is up 在拿到 done,但這個 case 的執行結果會先拿到 done 再拿到 time is up,這樣的優點是你可以一次同時處理很多事情,拿到結果後進行下一步,之前要達到這樣的效果,要寫很多程式碼,然而現實生活中其實大多的功能都必須依序執行,所以有了 promise 來解決這件事情,網路上範例很多,我這邊會用最精簡的作法當作是個人的筆記。

以上方例子來說,修改方式如下:

const runTimeout = async (time) => {
    return new Promise((resolve, reject) => {
        setTimeout(() => {
            resolve(time);
        }, time);
    });
};

const main = async () => {
    try {
        const r1 = await runTimeout(2000);
        const r2 = await runTimeout(1000);

        console.log(r1);
        console.log(r2);
        console.log('done');
    } catch (e) {
        console.error(e);
    }
};

main();

我將程式買改寫成可帶入參數,這樣可以驗證他的確是依序做完的,執行結果為:

2000
1000
done

另一個情境是,執行數量是動態的話要怎麼實現,範例如下:

const runTimeout = async (time) => {
    return new Promise((resolve, reject) => {
        setTimeout(() => {
            resolve(time);
        }, time);
    });
};

const times = [
    3000,
    1000,
    2000
];

const main = async () => {
    try {
        Promise.all(times.map(runTimeout)).then((msg) => {
            console.log(msg);
            console.log('done');
        });
    } catch (e) {
        console.error(e);
    }
};

main();

執行結果:

[ 3000, 1000, 2000 ]
done

第二種作法:

const runTimeout = async (time, name) => {
    return new Promise((resolve, reject) => {
        setTimeout(() => {
            resolve(`run ${name} in ${time} secs`);
        }, time);
    });
};

const times = [
    [3000, 'a'],
    [1000, 'b'],
    [2000, 'c']
];
const runners = times.map(async (item) => {
    const result = await runTimeout(item[0], item[1]);

    return result;
});
const main = async () => {
    try {
        Promise.all(runners)
            .then((msg) => {
                console.log(msg);
                console.log('done');
            });
    } catch (e) {
        console.error(e);
    }
};

main();

執行結果:

[ 'run a in 3000 secs',
  'run b in 1000 secs',
  'run c in 2000 secs' ]
done

2017/12/24

tmux

Common Shortcuts

<prefix> means ctrl + b.

# start new session
$ tmux new -s myname

# list sessions
$ tmux ls

# attach to session
$ tmux a -t 0

# rename session
$ tmux rename-session -t 0 NEW_NAME

# kill session
$ tmux kill-session -t 0

Windows Control

  • <prefix> c — Create window
  • <prefix> w — List windows
  • <prefix> n — Next window
  • <prefix> p — Previous window
  • <prefix> f — Find window
  • <prefix> , — Name window
  • <prefix> & — Kill window
  • <prefix> [ — Check history

Pane Control

  • <prefix> % — Vertical split
  • <prefix> " — Horizontal split
  • <prefix> o — Swap panes
  • <prefix> q — Show pane numbers
  • <prefix> x — Kill pane
  • <prefix> q — Show pane numbers (press number key to switch to that pane)
  • <prefix> { — Move the current pane left
  • <prefix> } — Move the current pane right
  • <prefix> z — Toggle pane zoom

Resize Pane

  • <prefix> : resize-pane -D (Resizes the current pane down)
  • <prefix> : resize-pane -U (Resizes the current pane upward)
  • <prefix> : resize-pane -L (Resizes the current pane left)
  • <prefix> : resize-pane -R (Resizes the current pane right)
  • <prefix> : resize-pane -D 20 (Resizes the current pane down by 20 cells)
  • <prefix> : resize-pane -U 20 (Resizes the current pane upward by 20 cells)
  • <prefix> : resize-pane -L 20 (Resizes the current pane left by 20 cells)
  • <prefix> : resize-pane -R 20 (Resizes the current pane right by 20 cells)
  • <prefix> : resize-pane -t 2 20 (Resizes the pane with the id of 2 down by 20 cells)
  • <prefix> : resize-pane -t -L 20 (Resizes the pane with the id of 2 left by 20 cells)

Start With Configuration

#!/bin/bash
tmux -2 new-session \; \
send-keys ''cd /var/www/test; vim'' C-m \; \
split-window -h \; \
send-keys 'top''top' C-m \; \
resize-pane -R 30 \; \
split-window -v \; \
send-keys ''while true; do date; sleep 1; done'' C-m \; \

My Config

套件

  • tpm 套件管理
  • tmux-pain-control 提供便捷的快捷鍵來控制 tmux 面板的導航與大小調整
  • tmux-sensible 提供一組合理的預設設置,讓 tmux 更加易於使用,特別是對新手友好
  • tmux-copycat 讓你在 tmux 中進行文字搜尋,並能夠快速複製選中的文字

安裝套件管理

git clone https://github.com/tmux-plugins/tpm.git ~/.tmux/plugins/tpm

~/.tmux.conf

# ~/.tmux.conf
set -g default-terminal "xterm"
setw -g mode-keys vi
set -g pane-border-status top
set -g mouse on

# List of plugins
set -g @plugin 'tmux-plugins/tpm'
set -g @plugin 'dracula/tmux'
set -g @dracula-plugins "git cpu-usage ram-usage time"
set -g @dracula-show-powerline true
set -g @plugin 'tmux-plugins/tmux-sensible'
set -g @plugin 'tmux-plugins/tmux-pain-control'
set -g @plugin 'tmux-plugins/tmux-copycat'

bind-key -T copy-mode-vi 'v' send -X begin-selection

run '~/.tmux/plugins/tpm/tpm'

~/.bashrc

# ~/.bashrc
alias tmux='tmux -2'

開啟 tmux 後執行 <prefix> + I。

Auto Start Config

#!/bin/sh

SERVICE=apache_watcher

tmux has-session -t $SERVICE

if [[ $? != 0 ]]; then
	tmux new -s $SERVICE -d

	tmux split-window -h
	tmux split-window -v -t $SERVICE:0.1
	tmux split-window -v -t $SERVICE:0.0

	tmux send-keys -t $SERVICE:0.0 'htop'  C-m
	tmux send-keys -t $SERVICE:0.2 'nload' C-m
	tmux send-keys -t $SERVICE:0.3 'sudo apachetop' C-m

	tmux clock-mode -t $SERVICE:0.1
	tmux resize-pane -R 30
fi

tmux a -t $SERVICE

Interesting Command

  • <prefix> : select-pane -T “NAME” - Set the title of the active pane
  • <prefix> : select-layout even-horizontal - Make horizontal pane widths equal
  • <prefix> : select-layout even-vertical - Make vertical pane heights equal
  • <prefix> : select-layout tiled - Arrange panes in a tiled layout
  • <prefix> space - Cycle through available pane layouts
  • <prefix> alt + arrow - Resize pane in the arrow’s direction
  • <prefix> : resize-pane -x 100 - Set the pane width to 100 cells
  • <prefix> : resize-pane -y 100 - Set the pane height to 100 cells

Reference Site

https://gist.github.com/MohamedAlaa/2961058
https://blog.htbaa.com/news/tmux-scripting

2017/12/18

Simple Regular Expression In PHP, Python, Node.js

今天需要用正規式找出符合 xxx[yyy] 後面配任意內容的結果,紀錄一下三種語言怎麼操作正規式。

PHP
<?php

$pattern = '/\w+\[\w+\].*/i';
$strings = [
    'aaa[bbb]',
    '1234[bbb][cccc]',
    '[111][22]',
    'dd[gg]ff',
    'aa[',
    'aaa][bbbb',
    'aaabbbccc',
];

foreach ($strings as $s) {
    var_dump($s.' is '.preg_match($pattern, $s));
}
Python
import re

pattern = r'\w+\[\w+\].*';
strings = [
    'aaa[bbb]',
    '1234[bbb][cccc]',
    '[111][22]',
    'dd[gg]ff',
    'aa[',
    'aaa][bbbb',
    'aaabbbccc'
];

for s in strings:
    print('%s is %s' % (s, True if re.match(pattern, s, re.IGNORECASE) else False))
Node.js
const pattern = /\w+\[\w+\].*/i;
const strings = [
    'aaa[bbb]',
    '1234[bbb][cccc]',
    '[111][22]',
    'dd[gg]ff',
    'aa[',
    'aaa][bbbb',
    'aaabbbccc'
];

for (v of strings) {
    console.log(v, 'is', pattern.test(v));
}

2017/11/06

crawler in PHP, Python, Node.js

網頁爬蟲是滿常遇到的課題,今天練習一下用這三種語言寫爬蟲

PHP

sunra/php-simple-html-dom-parser

crawler.php
<?php

include 'vendor/autoload.php';

use Sunra\PhpSimple\HtmlDomParser;

$url = 'https://www.ptt.cc/bbs/NBA/index.html';
$html = HtmlDomParser::file_get_html($url);
$titles = $html->find('div.title');
$file = fopen('php.txt', 'w');

foreach ($titles as $title) {
    $subject = trim($title->plaintext)."\n";
    fwrite($file, $subject);
}

fclose($file);
$html->clear();

Python

crawler.py
# -*- coding: utf-8 -*-

import sys
import requests
from bs4 import BeautifulSoup

reload(sys)
sys.setdefaultencoding('utf-8')

url = 'https://www.ptt.cc/bbs/NBA/index.html'
res = requests.get(url)
soup = BeautifulSoup(res.text, 'html.parser')
titles = soup.find_all('div', 'title')
f = open('py.txt', 'w');

for title in titles:
    f.write('%s\n' % title.text.strip())

f.close()

Node.js

crawler.js
const cheerio = require('cheerio');
const request = require('request');
const fs = require('fs');

const url = 'https://www.ptt.cc/bbs/NBA/index.html';
const out = fs.createWriteStream('nodejs.txt');

request(url, (err, res, body) => {
    if (err) {
        console.log(err);
    }

    const $ = cheerio.load(body);
    const divs = $('div.title');

    divs.each((i, item) => {
        out.write($(item).text().trim() + '\n');
    });

    out.end();
});

這三個程式執行的結果會得到一樣的內容,當然這是很簡略的範例,照理說要寫一些容錯判斷,但練習用的範例先這樣吧,再附上執行結果比較一下速度

PHP 7.0.22
$ time php crawler.php

real    0m0.614s
user    0m0.072s
sys     0m0.032s

real    0m0.570s
user    0m0.056s
sys     0m0.032s

real    0m0.570s
user    0m0.056s
sys     0m0.032s

real    0m0.515s
user    0m0.040s
sys     0m0.032s

real    0m0.515s
user    0m0.040s
sys     0m0.032s

real    0m0.515s
user    0m0.040s
sys     0m0.032s
Node.js v9.0.0
$ time node crawler.js

real    0m1.357s
user    0m0.244s
sys     0m0.396s

real    0m1.744s
user    0m0.376s
sys     0m0.312s

real    0m1.312s
user    0m0.272s
sys     0m0.380s

real    0m1.522s
user    0m0.348s
sys     0m0.304s

real    0m1.369s
user    0m0.288s
sys     0m0.396s

real    0m1.523s
user    0m0.388s
sys     0m0.344s
Python 2.7.12
$ time python crawler.py
real    0m0.666s
user    0m0.232s
sys     0m0.064s

real    0m0.682s
user    0m0.204s
sys     0m0.088s

real    0m0.709s
user    0m0.240s
sys     0m0.064s

real    0m0.624s
user    0m0.228s
sys     0m0.032s

real    0m0.669s
user    0m0.264s
sys     0m0.052s

real    0m0.698s
user    0m0.260s
sys     0m0.068s

基本上 PHP 跟 Python 都是按下去一下就做完了,Node.js 有等待的感覺,不知道是哪個環節出問題,我有把檔案寫入跟 dom 搜尋的部份關閉,速度沒差多少,所以是 [request][] 套件慢?但網路大家都推 [request][],他應該也是基於 node.js 內建的 http module 上的高級封裝,而且其他人也都有用套件,只能說幾乎我每次測試東西 node.js 數據都會吊車尾,目前還是選擇使用他好用的工具群為主。

2017/10/20

解構賦值比較

PHP 有一個可以替換變數的 function 叫 list,使用範例如下

test.php
$arrs = ['chan', 37];
list($name, $age) = $arrs;
var_dump($name, $age);
$ php test.php
string(4) "chan"
int(37)

在 node.js 的世界的話,用法是這樣的

test.js
const arrs = ['chan', 37];
const [name, age] = arrs;
console.log(name, age);
$ node test.js
chan 37

python 的話超簡單

test.py
arrs = ['chan', 37]
name, age = arrs
print(name, age)
$ python test.py
('chan', 37)

在 PHP 以及 js 的世界其實我自己用的不多,其他人的範例 code 也不算到常見,但在 python 的世界出現的頻率還算不低

2017/10/19

動態呼叫 module - python

在寫程式上我們會常遇到使用不同的參數動態的呼叫他的 class,在 PHP 會使用 call_user_fun 或 call_user_func_array 來達到這個目的,而 node.js 載入 module 本來就是可以使用變數所以相對簡單,舉例來說

tester.js
module.exports = {
    test() {
        console.log('test');
    }
};
main.js
moduleName = 'tester';
const tester = require(`./${moduleName}`);

tester.test();
$ node main.js
test

不過 python 載入模組的方式無法使用字串,但可以透過 __import__ 以及 getattr 這兩個方式組合達到我們要的目的,檔案的結構如下:

transportation/
	__init__.py
	base.py
	bike.py
	car.py
	main.py
base.py
class Base(object):
    def go(self):
        print('go with {excute_name}'.format(excute_name=self.excute_name))

這隻 module 簡單來講就是 car 以及 bike 共同繼承的工具,目的在測試這樣的載入方式使用繼承會不會有問題

car.py
from base import Base

class Car(Base):
    excute_name = 'car';

基本上只是初始化一個 module,放一個 property

bike.py
from base import Base

class Bike(Base):
    excute_name = 'bike'

同上

main.py
# -*- coding: utf-8 -*-

transportations = [
    'car',
    'nope',
    'bike'
]

for transportation in transportations:
    class_name = transportation.capitalize() # 字首改大寫即為 class name
    module_name = 'transportation.{name}'.format(name=transportation)

    try:
        module = __import__(module_name, globals(), locals(), [class_name])
        collector = getattr(module, class_name)()
        collector.go()
    except Exception as e:
        print(str(e))

我們把要呼叫的 module 寫成一個 array 跑迴圈,中間我故意放了一個不存在的 module,程式碼裡面有使用 try exception,所以不用怕出錯會中斷,執行程式結果如下

$ python main.py
go with car
No module named nope
go with bike

2017/08/07

Prompt In All Language

最近喜歡刻一些互動的小工具,程式碼將需求分析後在 terminal 與 user 互動,confirm 內容後動作是常見的,這邊紀錄一下三個語言的 prompt 方法

prompt.php
echo 'Are you sure? [y/n]: ';

$handle = fopen('php://stdin', 'r');
$answer = trim(fgets($handle));

if ($answer === 'y') {
    echo 'Yes!'.PHP_EOL;
} else {
    echo 'Canceled!'.PHP_EOL;
}

fclose($handle);
prompt.js
const readline = require('readline');

const rl = readline.createInterface({
    input: process.stdin,
    output: process.stdout
});

rl.question('Are you sure? [y/n]: ', (answer) => {
    if (answer.trim() === 'y') {
        console.log('Yes!');
    } else {
        console.log('Canceled!');
    }

    process.exit();
});
prompt.sh
#!/bin/bash

read -p 'Are you sure? [y/n]: ' answer

if [[ $answer == 'y' ]]; then
    echo "Yes!"
else
    echo "Canceled!"
fi