git » dnss » commit 337dedc

tests: Increase retries on the external tests

author Alberto Bertogli
2024-11-17 10:36:10 UTC
committer Alberto Bertogli
2024-11-17 10:58:29 UTC
parent 5a08a712bb212608bedf9699ed627959aae7b114
tests: Increase retries on the external tests

The external tests are heavily dependent on the testing environment,
because they reach out to live external resolvers.

Because of it, they are quite sensitive to transient failures, and this
is often experienced as flakiness in the Github CI.

To make these less flaky, this patch increases the retries, with
incremental delay between each attempt.
tests/external.sh +16 -8 
diff --git a/tests/external.sh b/tests/external.sh
index 1e87c59..0e601ea 100755
--- a/tests/external.sh
+++ b/tests/external.sh
@@ -227,15 +227,23 @@ do
 	echo "## DoH against $server"
 	dnss -enable_dns_to_https -dns_listen_addr "localhost:1053" \
 		-https_upstream "$server"
-	# Retry once after giving it some time, because the test environment
-	# and/or the server may be flaky.
-	if ! resolve; then
-		echo
-		echo "### Retrying in 1s"
-		sleep 1
-		resolve
-	fi
+
+	# Retry a few times with some delay in between, because the test
+	# environment and/or the server may be flaky.
+	success=0
+	for delay_sec in 0 1 3 5 10; do
+		sleep $delay_sec
+		if resolve; then
+			success=1
+			break
+		fi
+	done
+
 	kill $PID
+	if [ "$success" -ne 1 ]; then
+		echo "Failed to resolve with upstream $server"
+		exit 1
+	fi
 done