httpd tomcat

install apache on VM1

yum install -y httpd

start apache

sudo systemctl start httpd

starts at boot

sudo systemctl enable httpd

install apxs

yum install httpd-devel

  • find / -name apxs
  • which apxs (this file path will be used in ./configure below)

install gcc

yum install -y gcc*
yum install autoconf
yum install libtool

install mod_jk

wget http://www.eu.apache.org/dist/tomcat/tomcat-connectors/jk/tomcat-connectors-1.2.46-src.tar.gz
tar -xf tomcat-connectors-1.2.46-src.tar.gz
cd tomcat-connectors-1.2.46-src/native
./buildconf.sh
./configure –with-apxs=/usr/bin/apxs
make && make install
(find / -name mod_jk.so
/usr/lib64/httpd/modules/mod_jk.so)

Configure httpd.conf

Now that you’ve installed mod_jk, you need to configure it in your httpd.conf file so that you can use it.
You can find httpd.conf in the ‘conf’ directory of your Apache HTTPD home folder.
If httpd in installed by ‘yum install httpd’, this file is
/etc/httpd/conf/httpd.conf

mod_jk requires two entities:

  • mod_jk.xxx - The Apache HTTP Server module, depending on your operating system, it will be mod_jk.so, mod_jk.nlm or MOD_JK.SRVPGM (see the build section).
  • workers.properties - A file that describes the host(s) and port(s) used by the workers (Tomcat processes). A sample workers.properties can be found under the conf directory in the source download.

The configuration provided below, based on the Tomcat documentation, is the minimum configuration required for mod_jk to run correctly.

#Load mod_jk module
LoadModule jk_module /usr/lib64/httpd/modules/mod_jk.so
# Add the module (activate this lne for Apache 1.3)
# AddModule mod_jk.c
# Where to find workers.properties
JkWorkersFile /etc/httpd/conf/workers.properties
# Where to put jk shared memory
JkShmFile /var/log/httpd/mod_jk.shm
# Where to put jk logs
JkLogFile /var/log/httpd/mod_jk.log
# Set the jk log level [debug/error/info]
JkLogLevel info
# Send requests for context /examples to worker named worker1
# 至于为什么 balancer,https://tomcat.apache.org/connectors-doc/common_howto/loadbalancers.html
JkMount /playbook/* balancer

set up two tomcat on VM2, VM3

wget http://ftp.mirror.tw/pub/apache/tomcat/tomcat-8/v8.5.38/bin/apache-tomcat-8.5.38.tar.gz

Configure workers.properties

The workers.properties file defines a list of Tomcat ‘workers’ to which Apache HTTPD can pass requests.
8009 is the ajp port

1
2
3
4
5
worker.node1.host=192.168.77.131
worker.node1.port=8009

worker.node2.host=192.168.77.132
worker.node2.port=8009

test

start two tomcat
可以在两台tomcat里面都放一个playbook
第一台里面放一个webapp/playbook/1.txt, 内容是I am node1
第一台里面放一个webapp/playbook/1.txt, 内容是I am node2
systemctl restart httpd
现在访问
http://192.168.77.130/playbook/1.txt
会发现I am node1和I am node2交替出现
如果强制停止一台tomcat,整个服务不会中断

refer

systemctl start httpd
systemctl restart/start/status httpd
https://tomcat.apache.org/connectors-doc/webserver_howto/apache.html
https://www.mulesoft.com/tcat/apache-tomcat-mod-jk-connector-configuration
https://tomcat.apache.org/connectors-doc/common_howto/workers.html
https://tomcat.apache.org/connectors-doc/common_howto/loadbalancers.html

java log implemention

logback for spring data jpa

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
<configuration>
<appender name=”STDOUT” class=”ch.qos.logback.core.ConsoleAppender”>
<encoder class=”ch.qos.logback.classic.encoder.PatternLayoutEncoder”>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36}.%M\(%line\) — %msg%n
</pattern>
</encoder>
</appender>

<appender name=”FILE” class=”ch.qos.logback.core.rolling.RollingFileAppender”>
<file>${catalina.home}/logs/playbook.log</file>
<rollingPolicy class=”ch.qos.logback.core.rolling.TimeBasedRollingPolicy”>
<! — daily rollover →
<fileNamePattern>${catalina.home}/logs/playbook.%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy
class=”ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP”>
<! — or whenever the file size reaches 50MB →
<maxFileSize>50MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
<! — keep 30 days’ worth of history →
<maxHistory>30</maxHistory>
</rollingPolicy>
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36}.%M\(%line\) — %msg%n</pattern>
</encoder>
</appender>

<logger name=”ch.qos.logback” level=”WARN” />
<logger name=”com.demo.test” level=”DEBUG” />
<logger name=”org.springframework” level=”DEBUG” />
<logger name=”org.springframework.beans” level=”DEBUG” />
<logger name=”org.hibernate.SQL” level=”DEBUG” />
<logger name=”org.hibernate.type.descriptor.sql” level=”TRACE” />
<root level=”INFO”>
<appender-ref ref=”STDOUT” />
<appender-ref ref=”FILE” />
</root>
</configuration>

注意hibernate的log 需要 configure Hibernate to log via SLF4J,
System.setProperty(“org.jboss.logging.provider”, “slf4j”);
同时”hibernate.show_sql”设置为 “false”, 直接用logback,不要这种自带的控制台输出

tomcat log

所有的控制台输出都会重定向到tomcat/logs/catalina.out,但是maven tomcat7:run启动embended tomcat貌似没有这个文件
tocmat/logs/localhost.out 也会记录用户没有catch到的异常

reference

https://tomcat.apache.org/tomcat-8.5-doc/logging.html
https://stackoverflow.com/questions/55411338/why-the-hibernate-does-not-work-with-logback

AES base64

sample code

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
/**
https://aesencryption.net/
https://www.clickssl.net/blog/128-bit-ssl-encryption-vs-256-bit-ssl-encryption
https://www.quora.com/Is-AES256-more-secure-than-AES128-Whats-the-different
https://en.wikipedia.org/wiki/Advanced_Encryption_Standard
https://stackoverflow.com/questions/273396/aes-encryption-what-are-public-and-private-keys
*/

import sun.misc.BASE64Decoder;
import sun.misc.BASE64Encoder;

import java.nio.charset.StandardCharsets;
import javax.crypto.BadPaddingException;
import javax.crypto.Cipher;
import javax.crypto.IllegalBlockSizeException;
import javax.crypto.NoSuchPaddingException;
import javax.crypto.spec.SecretKeySpec;
import java.io.IOException;
import java.io.UnsupportedEncodingException;
import java.net.URLDecoder;
import java.net.URLEncoder;
import java.security.InvalidKeyException;
import java.security.Key;
import java.security.NoSuchAlgorithmException;

public class AESEncryptionUtil {

// http://www.allkeysgenerator.com/Random/Security-Encryption-Key-Generator.aspx
private static String key = “customized key”;
public static BASE64Encoder enc = new BASE64Encoder();
public static BASE64Decoder dec = new BASE64Decoder();


// eccrypt by AES, and encode by base64 to transfer by http
public static String encrypt(String clearText) throws Exception {
String encodedEncryptText = “”;
try {
Key aesKey = new SecretKeySpec(key.getBytes(), “AES”);
Cipher cipher = Cipher.getInstance(“AES”);

cipher.init(Cipher.ENCRYPT_MODE, aesKey);
byte[] encrypted = cipher.doFinal(clearText.getBytes(“UTF-8”));

encodedEncryptText = enc.encode(encrypted);

} catch (NoSuchAlgorithmException e) {
throw new Exception(e.getMessage());
} catch (NoSuchPaddingException e) {
throw new Exception(e.getMessage());
} catch (BadPaddingException e) {
throw new Exception(e.getMessage());
} catch (UnsupportedEncodingException e) {
throw new Exception(e.getMessage());
} catch (IllegalBlockSizeException e) {
throw new Exception(e.getMessage());
} catch (InvalidKeyException e) {
throw new Exception(e.getMessage());
}
return encodedEncryptText;

}

// encode by base64, decrypt by AES
public static String decrypt(String encodedEncryptText) throws Exception {
String decrypted = “”;
try {
Key aesKey = new SecretKeySpec(key.getBytes(), “AES”);

Cipher cipher = Cipher.getInstance(“AES”);
cipher.init(Cipher.DECRYPT_MODE, aesKey);

byte[] decodeBufferArray = dec.decodeBuffer(encodedEncryptText);

decrypted = new String(cipher.doFinal(decodeBufferArray), StandardCharsets.UTF_8);

} catch (NoSuchAlgorithmException e) {
throw new Exception(e.getMessage());
} catch (NoSuchPaddingException e) {
throw new Exception(e.getMessage());
} catch (BadPaddingException e) {
throw new Exception(e.getMessage());
} catch (IllegalBlockSizeException e) {
throw new Exception(e.getMessage());
} catch (InvalidKeyException e) {
throw new Exception(e.getMessage());
} catch (IOException e) {
throw new Exception(e.getMessage());
}
return decrypted;

}

public static void main(String[] args) {
String a = null;
try {
a = encrypt(“mengxu1128@gmail.com”);
} catch (Exception e) {
e.printStackTrace();
}
System.out.println(a);

String b = null;
try {
b = decrypt(a);

} catch (Exception e) {
e.printStackTrace();
}
System.out.println(b);
}
}

虚拟机增加空间

add space

我们用VMware创建好虚拟机之后可能会遇到空间不足的情况,这个时候需要扩展磁盘空间然后将新的空间挂载 到指定的目录

vm addspace

虚拟机设置里面先扩展好空间

fdisk -l查看磁盘占用情况
可以看到/dev下面的sda已经到几,比如我的机器已经到2,说明新创建的分区将会是sda3

输入fdisk /dev/sda,然后一步一步的回车,最后输入w保持修改
重启reboot
这时在/dev/目录下,才能看到了新的分区比如/dev/sda3
mkfs.ext2 /dev/sda3 格式化
在根目录下创建disk3 (mkdir /disk3)
mount /dev/sda3 /disk43/将分区mount到/disk3/
在vi修改/etc/fstab文件,加入 /dev/sda3 /disk3 ext2 defaults 0 0 一行,并保存,实现开机自动mount

至此,新增加的磁盘空间容量,即可在disk3上体现,并且重新开机自动mount该分区,追加磁盘空间的工作完毕。

reference

/dev/ is the part in the unix directory tree that contains all “device” files
So /dev/sda9 means the ninth partition on the first drive.

https://blog.csdn.net/lyd135364/article/details/78623119

mysql8 centos7 setup

mysql 8

Install MySQL YUM repository and mysql

Start MySQL server and autostart MySQL on boot

  • systemctl start mysqld.service ## use restart after update
  • systemctl enable mysqld.service

Get Your Generated Random root Password

1
grep 'A temporary password is generated for root@localhost' /var/log/mysqld.log |tail -1

Example Output:
2015-11-20T21:11:44.229891Z 1 [Note] A temporary password is generated for root@localhost: -et)QoL4MLid
And root password is: -et)QoL4MLid

Connect to MySQL database (localhost) with password

1
mysql -u root -p

会提示修改密码

1
ALTER USER 'root'@'localhost' IDENTIFIED BY 'Python3&&890';

但是至此是不能从远程访问此mysqld

enable root remote access

1
2
3
use mysql;
UPDATE user SET host = '%' WHERE user = 'root';
flush privileges;

create user and enable remote access

1
2
3
4
use mysql;
CREATE USER 'xuhang'@'%' IDENTIFIED BY 'Python3&&890'; -- 创建用户
GRANT ALL ON *.* TO 'xuhang'@'%' WITH GRANT OPTION; -- 授权
flush privileges;

backup data

1
2
3
4
5
// backup all db
mysqldump --all-databases -uroot -p> /tmp/dump.sql

// backup specific db
mysqldump --databases test1 -uroot -p> /tmp/dump.sql

recover data

1
2
mysql -uroot -p
source /tmp/dump.sql

reference

IO

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
import com.google.common.io.Files;

import java.io.*;
import java.nio.charset.Charset;
import java.nio.charset.StandardCharsets;

/**
* https://www.zhihu.com/question/27562173
* https://www.oracle.com/technetwork/articles/javase/supplementary-142654.html
*
* https://www.baeldung.com/java-write-to-file
*/
public class Test {
public static void main(String[] s) throws IOException {
// print os default charset
Charset.defaultCharset().name();
System.out.println( Charset.defaultCharset().name());


// below ascii charset will be garbled code.
FileOutputStream fos = new FileOutputStream("c:/temp/abc.txt");
fos.write("我的".getBytes(StandardCharsets.US_ASCII));

// below will be right
fos.write("中国".getBytes(StandardCharsets.UTF_8));


/**
*
* use below way to write string, no need to call getBytes
*/
Writer out = new BufferedWriter(new OutputStreamWriter(
new FileOutputStream("c:\\temp\\test.txt"), StandardCharsets.UTF_8));

out.append("Website UTF-8").append("\r\n");
out.append("你是 UTF-8").append("\r\n");
out.append("我的 UTF-8").append("\r\n");

out.flush();
out.close();


/**
* append a String to the existing file:
* use below way to write string, no need to call getBytes
* but this way can not specify charset
*/
Writer out2 = new BufferedWriter(new FileWriter("c:/temp/test2.txt", true));

out2.append("Website2 UTF-8").append("\r\n");
out2.append("你是2 UTF-8").append("\r\n");
out2.append("我的2 UTF-8").append("\r\n");

out2.flush();
out2.close();


// java 7 write string to file
Files.write("我的", new File("c:/temp/test3.txt"), StandardCharsets.UTF_8);

}
}

elasticsearch quickstart

Environment Setup

1
2
3
4
5
6
7
8
9
10
11
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.6.0.tar.gz
wget https://artifacts.elastic.co/downloads/kibana/kibana-6.6.0-linux-x86_64.tar.gz

# if you want to access kibana remotely, please update `server.host` to real IP in
# `kibana-6.6.0-linux-x86_64/config/kibana.yml` like 192.168.77.140

# You should run elasticsearch in non-root user
bin/elasticsearch -d
bin/kibana or `nohup bin/kibana &`

# now you can access http://192.168.77.140:5601

elasticsearch http verb

elasticsearch 按照HTTP协议下RestFul框架的交互格式就像这样:

1
2
3
4
POST /uri 创建
DELETE /uri/xxx 删除
PUT /uri/xxx 更新或创建
GET /uri/xxx 查看

sample case

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
PUT /customer?pretty

PUT /customer/_doc/1?pretty
{
"name": "John Doe"
}

GET /customer/_doc/1?pretty

DELETE /customer?pretty
GET /_cat/indices?v

PUT /customer/_doc/1?pretty
{
"name": "John Doe"
}

PUT /customer/_doc/1?pretty
{
"name": "Jane Doe"
}

PUT /customer/_doc/2?pretty
{
"name": "Jane Doe"
}

POST /customer/_doc?pretty
{
"name": "Jane Doe"
}

GET /customer/_search?q=Doe

GET /customer/_search
{
"query" : {
"match": { "name": "Doe" }
},
"highlight" : {
"fields" : {
"name" : {}
}
}
}

GET /_search
{
"query" : {
"match": { "name": "Doe" }
},
"highlight" : {
"fields" : {
"name" : {}
}
}
}

//这种更新是删除再增加,version变化了
PUT /customer/_doc/1?pretty
{
"name": "John Doe"
}

//这样更新是真的更新,version不变
POST /customer/_doc/1/_update?pretty
{
"doc": { "name": "Jane Doe", "age": 20 }
}

//get all index
curl "localhost:9200/_cat/indices?v"

curl -X GET "localhost:9200/bank/_search?q=*&sort=account_number:asc&pretty"
curl -X GET "localhost:9200/bank/_search?q=Fitzgerald&sort=account_number:asc&pretty"

//查询某个index
GET /twitter/_search
{
"query" : {
"term" : { "user" : "kimchy" }
}
}

// 查询所有的index
GET /_search
{
"from" : 0, "size" : 10,
"query" : {
"term" : { "user" : "kimchy" }
}
}

puppeteer centos7 setup

Install node, npm

1
2
3
4
5
6
// 直接下载编译好的node,编译这个非常费时间
wget https://nodejs.org/dist/v10.15.0/node-v10.15.0-linux-x64.tar.xz
tar xzf node-v10.15.0-linux-x64.tar.xz

// 设置好path
npm install -g yarn

安装好chrome在linux上需要的依赖

1
2
yum install pango.x86_64 libXcomposite.x86_64 libXcursor.x86_64 libXdamage.x86_64 libXext.x86_64 libXi.x86_64 libXtst.x86_64 cups-libs.x86_64 libXScrnSaver.x86_64 libXrandr.x86_64 GConf2.x86_64 alsa-lib.x86_64 atk.x86_64 gtk3.x86_64 -y
yum install ipa-gothic-fonts xorg-x11-fonts-100dpi xorg-x11-fonts-75dpi xorg-x11-utils xorg-x11-fonts-cyrillic xorg-x11-fonts-Type1 xorg-x11-fonts-misc -y

test

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
mkdir puppeteer-test
cd puppeteer-test
yarn add puppeteer // 下载失败可以提前执行 npm config set puppeteer_download_host=https://npm.taobao.org/mirrors

node test.js //test.js 内容如下

const puppeteer = require('puppeteer');

(async () => {
const browser = await puppeteer.launch({
args: ['--no-sandbox','--disable-setuid-sandbox'],


timeout: 15000,
//如果是访问https页面 此属性会忽略https错误
ignoreHTTPSErrors: true,
// 打开开发者工具, 当此值为true时, headless总为false
devtools: false,
// 关闭headless模式, 不会打开浏览器
headless: true
});
const page = await browser.newPage();
await page.goto('https://www.baidu.com');
await page.screenshot({path: 'example.png'});

await browser.close();
})();

references

// docker 中使用puppeteer
// docker中已经安装好所有的动态库,同时global的puppeteer也已经安装好
// docker会设置好ENV NODE_PATH=”/usr/local/share/.config/yarn/global/node_modules:${NODE_PATH}”
// https://serverfault.com/questions/594281/how-can-i-override-cmd-when-running-a-docker-image
https://github.com/alekzonder/docker-puppeteer

https://github.com/GoogleChrome/puppeteer/issues/1597
https://github.com/Googlechrome/puppeteer/issues/290

node.js module.export

module.export

The exports variable is available within a module’s file-level scope, and is assigned the value of module.exports before the module is evaluated.

It allows a shortcut, so that module.exports.f = … can be written more succinctly as exports.f = …. However, be aware that like any variable, if a new value is assigned to exports, it is no longer bound to module.exports:

1
2
module.exports.hello = true; // Exported from require of module
exports = { hello: false }; // Not exported, only available in the module

When the module.exports property is being completely replaced by a new object, it is common to also reassign exports:

1
2
3
module.exports = exports = function Constructor() {
// ... etc.
};

简单说就是exports如果被新的object覆盖掉了,那就必须是modules.exports = {}, 否则可以直接简写 exports.xxx={}

sample code

foo.js

1
2
const circle = require('./circle.js');
console.log(`The area of a circle of radius 4 is ${circle.area(4)}`);

circle.js

1
2
3
const { PI } = Math;
module.exports.area = (r) => PI * r ** 2; // the module can be ignored here
module.exports.circumference = (r) => 2 * PI * r;

上面这个例子里, 因为exports没有被赋新的object, 所以modules关键字可以省略

bar.js

1
2
3
const Square = require('./square.js');
const mySquare = new Square(2);
console.log(`The area of mySquare is ${mySquare.area()}`);

square.js

1
2
3
4
5
6
7
8
9
10
11
// assigning to exports will not modify module, must use module.exports
// the module can not be ignored here
module.exports = class Square {
constructor(width) {
this.width = width;
}

area() {
return this.width ** 2;
}
};

上面这个例子里, 因为exports被赋新的object, 所以modules关键字不可以省略