欢迎大家来到IT世界,在知识的湖畔探索吧!
今天本能是想测试一个PDO持久化,会不会带来会话混乱的问题 先贴一下PHP代码, 代码丑了点,但是坚持能run就行,反正就是做个测试。
<?php $dsn = 'mysql:host=localhost;dbname=test;charset=utf8'; $user = 'root'; $password = 'root'; // 设置 PDO 选项,启用持久化连接 $options = [ PDO::ATTR_PERSISTENT => true, PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION ]; try { // 创建持久化连接 $pdo = new PDO($dsn, $user, $password, $options); $stmt = $pdo->prepare("INSERT INTO test_last_insert_id (uni) VALUES (:uni);"); $uni = uniqid('', true); $stmt->bindValue(':uni', $uni); $aff = $stmt->execute(); // if ($aff === false) { throw new Exception("insert fail:"); } $id = $pdo->lastInsertId(); function getExecutedSql($stmt, $params) { $sql = $stmt->queryString; $keys = array(); $values = array(); // 替换命名占位符 :key with ? $sql = preg_replace('/\:(\w+)/', '?', $sql); // 绑定的参数可能包括命名占位符,我们需要将它们转换为匿名占位符 foreach ($params as $key => $value) { $keys[] = '/\?/'; $values[] = is_string($value) ? "'$value'" : $value; } // 替换占位符为实际参数 $sql = preg_replace($keys, $values, $sql, 1, $count); return $sql; } $stmt = $pdo->query("SELECT id FROM test_last_insert_id WHERE uni = '{$uni}'", PDO::FETCH_NUM); $row = $stmt->fetch(); $value = $row[0]; if ($value != $id) { throw new Exception("id is diff"); } echo "success" . PHP_EOL; } catch (PDOException $e) { header('HTTP/1.1 500 Internal Server Error'); file_put_contents('pdo_perisistent.log', $e->getMessage() . PHP_EOL); die('Database connection failed: ' . $e->getMessage()); } catch (Exception $e) { header('HTTP/1.1 500 Internal Server Error'); file_put_contents('pdo_perisistent.log', $e->getMessage() . PHP_EOL); die('Exception: ' . $e->getMessage()); }
欢迎大家来到IT世界,在知识的湖畔探索吧!
用wrk压测,一开始uniqid因为少了混淆参数还报了500,加了一下参数,用来保证uni值
欢迎大家来到IT世界,在知识的湖畔探索吧!% ./wrk -c100 -t2 -d3s --latency "http://localhost/pdo_perisistent.php" Running 3s test @ http://localhost/pdo_perisistent.php 2 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 52.17ms 7.48ms 103.38ms 80.57% Req/Sec 0.96k 133.22 1.25k 75.81% Latency Distribution 50% 51.06ms 75% 54.17ms 90% 59.45ms 99% 80.54ms 5904 requests in 3.10s, 1.20MB read Requests/sec: 1901.92 Transfer/sec: 397.47KB
1900 ~ 2600 之间的QPS,其实这个数值还是相当满意的,测试会话会不会混乱的问题也算完结了。 但是好奇心突起,之前一直没做过go和php执行sql下的对比,正好做一次对比压测
package main import ( "database/sql" "fmt" "net/http" "sync/atomic" "time" _ "github.com/go-sql-driver/mysql" "log" ) var id int64 = time.Now().Unix() * func generateUniqueID() int64 { return atomic.AddInt64(&id, 1) } func main() { dsn := "root:root@tcp(localhost:3306)/test?charset=utf8" db, err := sql.Open("mysql", dsn) if err != nil { log.Fatalf("Error opening database: %v", err) } defer func() { _ = db.Close() }() //// 设置连接池参数 //db.SetMaxOpenConns(100) // 最大打开连接数 //db.SetMaxIdleConns(10) // 最大空闲连接数 //db.SetConnMaxLifetime(time.Hour) // 连接最大存活时间 http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { var err error uni := generateUniqueID() // Insert unique ID into the database insertQuery := `INSERT INTO test_last_insert_id (uni) VALUES (?)` result, err := db.Exec(insertQuery, uni) if err != nil { log.Fatalf("Error inserting data: %v", err) } lastInsertID, err := result.LastInsertId() if err != nil { log.Fatalf("Error getting last insert ID: %v", err) } // Verify the last insert ID selectQuery := `SELECT id FROM test_last_insert_id WHERE uni = ?` var id int64 err = db.QueryRow(selectQuery, uni).Scan(&id) if err != nil { log.Fatalf("Error selecting data: %v", err) } if id != lastInsertID { log.Fatalf("ID mismatch: %d != %d", id, lastInsertID) } fmt.Println("success") }) _ = http.ListenAndServe(":8080", nil) }
truncate表压测结果,这低于预期了吧
欢迎大家来到IT世界,在知识的湖畔探索吧!% ./wrk -c100 -t2 -d3s --latency "http://localhost:8080" Running 3s test @ http://localhost:8080 2 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 54.05ms 36.86ms 308.57ms 80.77% Req/Sec 0.98k 243.01 1.38k 63.33% Latency Distribution 50% 43.70ms 75% 65.42ms 90% 99.63ms 99% 190.18ms 5873 requests in 3.01s, 430.15KB read Requests/sec: 1954.08 Transfer/sec: 143.12KB
开个连接池,清表再测,结果半斤八两
% ./wrk -c100 -t2 -d3s --latency "http://localhost:8080" Running 3s test @ http://localhost:8080 2 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 54.07ms 35.87ms 281.38ms 79.84% Req/Sec 0.97k 223.41 1.40k 60.00% Latency Distribution 50% 44.91ms 75% 66.19ms 90% 99.65ms 99% 184.51ms 5818 requests in 3.01s, 426.12KB read Requests/sec: 1934.39 Transfer/sec: 141.68KB
然后开启不清表的情况下,php和go的交叉压测
欢迎大家来到IT世界,在知识的湖畔探索吧!% ./wrk -c100 -t2 -d3s --latency "http://localhost:8080" Running 3s test @ http://localhost:8080 2 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 52.51ms 43.28ms 436.00ms 86.91% Req/Sec 1.08k 284.67 1.65k 65.00% Latency Distribution 50% 40.22ms 75% 62.10ms 90% 102.52ms 99% 233.98ms 6439 requests in 3.01s, 471.61KB read Requests/sec: 2141.12 Transfer/sec: 156.82KB % ./wrk -c100 -t2 -d3s --latency "http://localhost/pdo_perisistent.php" Running 3s test @ http://localhost/pdo_perisistent.php 2 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 41.41ms 10.44ms 77.04ms 78.07% Req/Sec 1.21k 300.99 2.41k 73.77% Latency Distribution 50% 38.91ms 75% 47.62ms 90% 57.38ms 99% 69.84ms 7332 requests in 3.10s, 1.50MB read Requests/sec: 2363.74 Transfer/sec: 493.98KB // 这里骤降是我很不理解的不明白是因为什么 % ./wrk -c100 -t2 -d3s --latency "http://localhost:8080" Running 3s test @ http://localhost:8080 2 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 156.72ms 75.48ms 443.98ms 66.10% Req/Sec 317.93 84.45 480.00 71.67% Latency Distribution 50% 155.21ms 75% 206.36ms 90% 254.32ms 99% 336.07ms 1902 requests in 3.01s, 139.31KB read Requests/sec: 631.86 Transfer/sec: 46.28KB % ./wrk -c100 -t2 -d3s --latency "http://localhost/pdo_perisistent.php" Running 3s test @ http://localhost/pdo_perisistent.php 2 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 43.47ms 10.04ms 111.41ms 90.21% Req/Sec 1.15k 210.61 1.47k 72.58% Latency Distribution 50% 41.17ms 75% 46.89ms 90% 51.27ms 99% 95.07ms 7122 requests in 3.10s, 1.45MB read Requests/sec: 2296.19 Transfer/sec: 479.87KB % ./wrk -c100 -t2 -d3s --latency "http://localhost:8080" Running 3s test @ http://localhost:8080 2 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 269.08ms 112.17ms 685.29ms 73.69% Req/Sec 168.22 125.46 520.00 79.59% Latency Distribution 50% 286.58ms 75% 335.40ms 90% 372.61ms 99% 555.80ms 1099 requests in 3.02s, 80.49KB read Requests/sec: 363.74 Transfer/sec: 26.64KB % ./wrk -c100 -t2 -d3s --latency "http://localhost/pdo_perisistent.php" Running 3s test @ http://localhost/pdo_perisistent.php 2 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 41.74ms 9.67ms 105.86ms 91.72% Req/Sec 1.20k 260.04 2.24k 80.33% Latency Distribution 50% 38.86ms 75% 46.77ms 90% 49.02ms 99% 83.01ms 7283 requests in 3.10s, 1.49MB read Requests/sec: 2348.07 Transfer/sec: 490.71KB % ./wrk -c100 -t2 -d3s --latency "http://localhost:8080" Running 3s test @ http://localhost:8080 2 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 464.85ms 164.66ms 1.06s 71.97% Req/Sec 104.18 60.01 237.00 63.16% Latency Distribution 50% 467.00ms 75% 560.54ms 90% 660.70ms 99% 889.86ms 605 requests in 3.01s, 44.31KB read Requests/sec: 200.73 Transfer/sec: 14.70KB % ./wrk -c100 -t2 -d3s --latency "http://localhost/pdo_perisistent.php" Running 3s test @ http://localhost/pdo_perisistent.php 2 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 50.62ms 9.16ms 85.08ms 75.74% Req/Sec 0.98k 170.66 1.30k 69.35% Latency Distribution 50% 47.93ms 75% 57.20ms 90% 61.76ms 99% 79.90ms 6075 requests in 3.10s, 1.24MB read Requests/sec: 1957.70 Transfer/sec: 409.13KB % ./wrk -c100 -t2 -d3s --latency "http://localhost:8080" Running 3s test @ http://localhost:8080 2 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 568.84ms 160.91ms 1.04s 66.38% Req/Sec 81.89 57.59 262.00 67.27% Latency Distribution 50% 578.70ms 75% 685.85ms 90% 766.72ms 99% 889.39ms 458 requests in 3.01s, 33.54KB read Requests/sec: 151.91 Transfer/sec: 11.13KB
go 的代码随着不断的测试,很明显处理速度在不断的下降,这说实话有点超出我的认知了。 PHP那边却是基本稳定的,go其实一开始我还用gin测试过,发现测试结果有点超出预料,还改了用http库来测试,这结果属实差强人意了。
突然明白之前经常看到别人在争论性能问题的时候,为什么总有人强调PHP性能并不差。 或许PHP因为fpm的关系导致每次加载大量文件导致的响应相对较慢,比如框架laravel 那个QPS只有一两百的家伙,但其实这个问题要解决也是可以解决的,也用常驻内存的方式就好了。再不行还有phalcon
我一直很好奇一直说PHP性能问题的到底是哪些人, 不会是从PHP转到其他语言的吧。
% php -v PHP 8.3.12 (cli) (built: Sep 24 2024 18:08:04) (NTS) Copyright (c) The PHP Group Zend Engine v4.3.12, Copyright (c) Zend Technologies with Xdebug v3.3.2, Copyright (c) 2002-2024, by Derick Rethans with Zend OPcache v8.3.12, Copyright (c), by Zend Technologies % go version go version go1.23.1 darwin/amd64
欢迎大家来到IT世界,在知识的湖畔探索吧!
这结果,其实不太能接受,甚至都不知道原因出在哪了,有大佬可以指出问题一下吗
加一下时间打印再看看哪里的问题
欢迎大家来到IT世界,在知识的湖畔探索吧!package main import ( "database/sql" "fmt" "net/http" "sync/atomic" "time" _ "github.com/go-sql-driver/mysql" "log" ) var id int64 = time.Now().Unix() * func generateUniqueID() int64 { return atomic.AddInt64(&id, 1) } func main() { dsn := "root:root@tcp(localhost:3306)/test?charset=utf8" db, err := sql.Open("mysql", dsn) if err != nil { log.Fatalf("Error opening database: %v", err) } defer func() { _ = db.Close() }() // 设置连接池参数 db.SetMaxOpenConns(100) // 最大打开连接数 db.SetMaxIdleConns(10) // 最大空闲连接数 db.SetConnMaxLifetime(time.Hour) // 连接最大存活时间 http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) { reqStart := time.Now() var err error uni := generateUniqueID() start := time.Now() // Insert unique ID into the database insertQuery := `INSERT INTO test_last_insert_id (uni) VALUES (?)` result, err := db.Exec(insertQuery, uni) fmt.Printf("insert since: %v uni:%d \n", time.Since(start), uni) if err != nil { log.Fatalf("Error inserting data: %v", err) } lastInsertID, err := result.LastInsertId() if err != nil { log.Fatalf("Error getting last insert ID: %v", err) } selectStart := time.Now() // Verify the last insert ID selectQuery := `SELECT id FROM test_last_insert_id WHERE uni = ?` var id int64 err = db.QueryRow(selectQuery, uni).Scan(&id) fmt.Printf("select since:%v uni:%d \n", time.Since(selectStart), uni) if err != nil { log.Fatalf("Error selecting data: %v", err) } if id != lastInsertID { log.Fatalf("ID mismatch: %d != %d", id, lastInsertID) } fmt.Printf("success req since:%v uni:%d \n", time.Since(reqStart), uni) }) _ = http.ListenAndServe(":8080", nil) }
截取了后面的一部分输出,这不会是SQL库的问题吧,
success req since:352.ms uni:00652 insert since: 163.ms uni:00688 insert since: 154.ms uni:00691 insert since: 158.094503ms uni:00689 insert since: 136.ms uni:00697 insert since: 141.ms uni:00696 insert since: 128.ms uni:00702 select since:412.94524ms uni:00634 success req since:431.ms uni:00634 select since:459.ms uni:00601 success req since:568.ms uni:00601 insert since: 134.39147ms uni:00700 select since:390.ms uni:00643 success req since:391.ms uni:00643 select since:366.098937ms uni:00648 success req since:373.ms uni:00648 insert since: 136.ms uni:00699 select since:420.ms uni:00640 success req since:425.ms uni:00640 insert since: 167.ms uni:00690 select since:272.22808ms uni:00671
单次请求的时候输出结果是符合预期的, 但是并发SQL时会出现执行慢的问题,这就很奇怪了
欢迎大家来到IT世界,在知识的湖畔探索吧!% curl localhost:8080 insert since: 1.ms uni:00703 select since:21.031284ms uni:00703 success req since:22.62274ms uni:00703
经群友提示还和唯一键的区分度有关,两边算法一致有点太难了,Go换了雪法ID之后就正常了。 因为之前 Go这边生成的uni值是递增的导致区分度很低,最终导致并发写入查询效率变低。
% ./wrk -c100 -t2 -d3s --latency "http://localhost:8080" Running 3s test @ http://localhost:8080 2 threads and 100 connections Thread Stats Avg Stdev Max +/- Stdev Latency 44.51ms 24.87ms 187.91ms 77.98% Req/Sec 1.17k 416.31 1.99k 66.67% Latency Distribution 50% 37.46ms 75% 54.55ms 90% 80.44ms 99% 125.72ms 6960 requests in 3.01s, 509.77KB read Requests/sec: 2316.02 Transfer/sec: 169.63KB
2024-12-02 更新
今天本来是想验证一下有关,并发插入自增有序的唯一键高延迟的问题,发现整个有问题的只有一行代码。 就是在查询时,类型转换的问题,插入和查询都转换之后,空表的情况下QPS 可以到4000多。即使在已有大数据量(几十万)的情况也有两千多的QPS。 现在又多了一个问题,为什么用雪花ID时不会有这样的问题。雪花ID也是int64类型的,这是为什么呢。
欢迎大家来到IT世界,在知识的湖畔探索吧!// 旧代码 err = db.QueryRow(selectQuery, uni).Scan(&id) if err != nil { log.Fatalf("Error selecting data: %v", err) } // 新代码 变化只有一个就是把uni 转成字符串之后就没有问题了 var realId int64 err = db.QueryRow(selectQuery, fmt.Sprintf("%d", uni)).Scan(&realId) if err != nil { log.Fatalf("Error selecting data: %v", err)
免责声明:本站所有文章内容,图片,视频等均是来源于用户投稿和互联网及文摘转载整编而成,不代表本站观点,不承担相关法律责任。其著作权各归其原作者或其出版社所有。如发现本站有涉嫌抄袭侵权/违法违规的内容,侵犯到您的权益,请在线联系站长,一经查实,本站将立刻删除。 本文来自网络,若有侵权,请联系删除,如若转载,请注明出处:https://itzsg.com/107735.html