From 2bdcb7159b1ac097355e95864e979b4f68bc1a4e Mon Sep 17 00:00:00 2001

From: =?UTF-8?q?Petr=20Men=C5=A1=C3=ADk?= <pemensik@redhat.com>

Date: Thu, 7 Nov 2019 14:31:03 +0100

Subject: [PATCH] Implement serve-stale in 9.11

MIME-Version: 1.0

Content-Type: text/plain; charset=UTF-8

Content-Transfer-Encoding: 8bit

Squashed commit of the following:

commit 32f47f36e545223b2a4757588d7bd4af8c5f5760

Author: Petr Menšík <pemensik@redhat.com>

Date:   Tue Sep 3 18:45:54 2019 +0200

    convert serve_stale to db_test

    Manual checkout from commit e8f61dd315c5d1c88915bb79361182241e42e47a.

    Use test modified for cmocka, including serve-stale tests.

commit 071eb1fb0786f6d614955813d99c3caabff33383

Author: Michał Kępień <michal@isc.org>

Date:   Fri Apr 27 09:13:26 2018 +0200

    Detect recursion loops during query processing

    Interrupt query processing when query_recurse() attempts to ask the same

    name servers for the same QNAME/QTYPE tuple for two times in a row as

    this indicates that query processing may be stuck for an indeterminate

    period of time, e.g. due to interactions between features able to

    restart query_lookup().

    (cherry picked from commit 46bb4dd124ed031d4c219d1e37a3c6322092e30c)

commit c12090bc361c7fa4522ace73899e778e44e9b295

Author: Petr Menšík <pemensik@redhat.com>

Date:   Mon Sep 2 11:12:32 2019 +0200

    Fix test name used in whole test-suite

    Correct name is serve-stale

commit ff4d826f295d268a248ca06941d65c903e1b405c

Author: Petr Menšík <pemensik@redhat.com>

Date:   Fri Aug 30 17:43:28 2019 +0200

    Clean files in more generic rules

commit 8d81ed15eda9a2a11e1433d1fdddacfc772708b6

Author: Petr Menšík <pemensik@redhat.com>

Date:   Thu Aug 29 21:27:57 2019 +0200

    [rt46602] Pass port numbers to tests via environment variables

    Manually applied commit f5d8f079008b648d2e343543e66dd728054c6101

commit 94fafa477891576286def8c4041ad127734af2d1

Author: Tony Finch <dot@dotat.at>

Date:   Tue Apr 10 16:17:57 2018 +0100

    Move serve-stale logging to its own category, so that its verbosity can be curtailed.

    (cherry picked from commit 4b442c309dfb2c8880b19af4133047655bb734df)

commit e0c884bee98c3d2533dfaa667f58c6a80d8a3a00

Author: Michał Kępień <michal@isc.org>

Date:   Fri Apr 27 09:13:26 2018 +0200

    Prevent check_stale_header() from leaking rdataset headers

    check_stale_header() fails to update the pointer to the previous header

    while processing rdataset headers eligible for serve-stale, thus

    enabling rdataset headers to be leaked (i.e. disassociated from a node

    and left on the relevant TTL heap) while iterating through a node.  This

    can lead to several different assertion failures.  Add the missing

    pointer update.

    (cherry picked from commit 391fac1fc8d2e470287b5cc4344b3adb90c6f54a)

commit d724cc1d80ee8d46113eaf82549d49636739b67c

Author: Matthijs Mekking <matthijs@isc.org>

Date:   Thu Jan 24 10:24:44 2019 +0100

    Print in dump-file stale ttl

    This change makes rndc dumpdb correctly print the "; stale" line.

    It also provides extra information on how long this data may still

    be served to clients (in other words how long the stale RRset may

    still be used).

    (cherry picked from commit 924ebc605db798e2a383ee5eaaebad739e7c789c)

commit 625da4bd4590ac6108bb30eddd23ceffb245ae49

Author: Michał Kępień <michal@isc.org>

Date:   Mon Oct 22 15:26:45 2018 +0200

    Check serve-stale behavior with a cold cache

    Ensure that serve-stale works as expected when returning stale answers

    is enabled, the authoritative server does not respond, and there is no

    cached answer available.

    (cherry picked from commit 27cfe83a388147edfa0451b28c06c746912ea684)

commit d67ae10461c409fdafdbbe64f857db2552b71059

Author: Michał Kępień <michal@isc.org>

Date:   Mon Oct 22 15:26:45 2018 +0200

    Check TTL of stale answers

    Make sure that stale answers returned when the serve-stale feature is

    enabled have a TTL matching the value of the stale-answer-ttl setting.

    (cherry picked from commit 893ab37ce78c658215bd3a019f25afe795b37d5a)

commit 50459107805e68e4a63a8e497bf58ef3ce013ddb

Author: Michał Kępień <michal@isc.org>

Date:   Mon Jul 9 14:35:12 2018 +0200

    Do not use Net::DNS::Nameserver in the "serve-stale" system test

    Net::DNS versions older than 0.67 respond to queries sent to a

    Net::DNS::Nameserver even if its ReplyHandler returns undef.  This makes

    the "serve-stale" system test fail as it takes advantage of the newer

    behavior.  Since the latest Net::DNS version available with stock

    RHEL/CentOS 6 packages is 0.65 and we officially support that operating

    system, bin/tests/system/serve-stale/ans2/ans.pl should behave

    consistently for various Net::DNS versions.  Ensure that by reworking it

    so that it does not use Net::DNS::Nameserver.

    (cherry picked from commit c4209418a50c09142375f7edadca731c526f3d3a)

commit 4b5befc714bb386bd245b1c14ce3bce5ae6fb5fa

Author: Petr Menšík <pemensik@redhat.com>

Date:   Tue Jun 5 21:38:29 2018 +0200

    Fix server-stale requirement, skip without Time::HiRes

    (cherry picked from commit 7a0c7bf9c8e6a724e52635eed213ad25b9504e66)

commit 5ce51a3a7e5ef3087c4d022e3fca42fb2fd0c996

Author: Ondřej Surý <ondrej@sury.org>

Date:   Wed Oct 18 13:01:14 2017 +0200

    [rt46602] Update server-stale test to run on port passed from run.sh script

    (cherry picked from commit f83ebd34b9555a5a834c58146035173bcbd01dda)

commit 3954a9bf3437f6fab050294a7f2f954a23d161ec

Author: Ondřej Surý <ondrej@sury.org>

Date:   Wed Oct 18 14:18:59 2017 +0200

    [rt46602] Add serve-stale working files to .gitignore

    (cherry picked from commit cba162e70e7fac43435a606106841a69ce468526)

commit 112aa21f5fa875494820e4d1eb70e41e10e1aae7

Author: Mark Andrews <marka@isc.org>

Date:   Thu Oct 12 15:33:47 2017 +1100

    test for Net::DNS::Nameserver

    (cherry picked from commit 5b60d0608ac2852753180b762d1917163f9dc315)

commit 9d610e46af8a636f44914cee4cf8b2016054db1e

Author: Mark Andrews <marka@isc.org>

Date:   Thu Oct 12 15:19:45 2017 +1100

    add Net::DNS prerequiste test

    (cherry picked from commit fa644181f51559da3e3913acd72dbc3f6d916e71)

commit e4ea7ba88d9a9a0c79579400c68a5dabe03e8572

Author: Mark Andrews <marka@isc.org>

Date:   Wed Sep 6 19:26:10 2017 +1000

    add quotes arount $send_response

    (cherry picked from commit 023ab19634b287543169e9b7b5259f3126cd60ff)

commit 0af0c5d33c2de34da164571288b650282c6be10a

Author: Mark Andrews <marka@isc.org>

Date:   Thu Nov 23 16:11:49 2017 +1100

    initalise serve_stale_ttl

    (cherry picked from commit 2f4e0e5a81278f59037bf06ae99ff52245cd57e9)

commit fbadd90ee81863d617c4c319d5f0079b877fe102

Author: Evan Hunt <each@isc.org>

Date:   Thu Sep 14 11:48:21 2017 -0700

    [master] add thanks to APNIC and add missing note for serve-stale

commit deb8adaa59955970b9d2f2fe58060a3cbf08312b

Author: Mark Andrews <marka@isc.org>

Date:   Wed Sep 6 12:16:10 2017 +1000

    silence 'staleanswersok' may be used uninitialized in this function warning. [RT #14147

commit 0e2d03823768dc545015e6ce309777210f4a9f85

Author: Petr Menšík <pemensik@redhat.com>

Date:   Thu Aug 29 19:57:58 2019 +0200

    More fixes to merge

commit 360e25ffe7623ea0a2eec49395001f4940967776

Author: Mark Andrews <marka@isc.org>

Date:   Wed Sep 6 09:58:29 2017 +1000

    4700.   [func]          Serving of stale answers is now supported. This

                            allows named to provide stale cached answers when

                            the authoritative server is under attack.

                            See max-stale-ttl, stale-answer-enable,

                            stale-answer-ttl. [RT #44790]

Signed-off-by: Petr Menšík <pemensik@redhat.com>

---

 bin/named/config.c                            |   9 +-

 bin/named/control.c                           |   2 +

 bin/named/include/named/control.h             |   1 +

 bin/named/include/named/log.h                 |   1 +

 bin/named/include/named/query.h               |  15 +

 bin/named/include/named/server.h              |  13 +-

 bin/named/log.c                               |   1 +

 bin/named/query.c                             | 164 +++++-

 bin/named/server.c                            | 177 +++++-

 bin/named/statschannel.c                      |   6 +

 bin/rndc/rndc.c                               |   2 +

 bin/rndc/rndc.docbook                         |  19 +

 bin/tests/system/chain/prereq.sh              |   7 +

 bin/tests/system/conf.sh.in                   |   2 +-

 bin/tests/system/dyndb/driver/db.c            |   2 +

 bin/tests/system/serve-stale/.gitignore       |  11 +

 bin/tests/system/serve-stale/ans2/ans.pl.in   | 178 ++++++

 bin/tests/system/serve-stale/clean.sh         |  15 +

 .../system/serve-stale/ns1/named1.conf.in     |  35 ++

 .../system/serve-stale/ns1/named2.conf.in     |  35 ++

 bin/tests/system/serve-stale/ns1/root.db      |   5 +

 .../system/serve-stale/ns3/named.conf.in      |  35 ++

 bin/tests/system/serve-stale/prereq.sh        |  38 ++

 bin/tests/system/serve-stale/setup.sh         |  13 +

 bin/tests/system/serve-stale/tests.sh         | 536 ++++++++++++++++++

 doc/arm/Bv9ARM-book.xml                       |  69 ++-

 doc/arm/logging-categories.xml                |  11 +

 doc/arm/notes-rh-changes.xml                  |  14 +-

 doc/misc/options                              |  10 +

 lib/bind9/check.c                             |  78 ++-

 lib/dns/cache.c                               |  38 +-

 lib/dns/db.c                                  |  22 +

 lib/dns/ecdb.c                                |   4 +-

 lib/dns/include/dns/cache.h                   |  21 +

 lib/dns/include/dns/db.h                      |  35 ++

 lib/dns/include/dns/rdataset.h                |  11 +

 lib/dns/include/dns/resolver.h                |  43 +-

 lib/dns/include/dns/types.h                   |   6 +

 lib/dns/include/dns/view.h                    |   3 +

 lib/dns/master.c                              |  14 +-

 lib/dns/masterdump.c                          |  23 +

 lib/dns/rbtdb.c                               | 207 ++++++-

 lib/dns/resolver.c                            |  78 ++-

 lib/dns/sdb.c                                 |   4 +-

 lib/dns/sdlz.c                                |   4 +-

 lib/dns/tests/db_test.c                       | 198 ++++++-

 lib/dns/view.c                                |   3 +

 lib/isccfg/namedconf.c                        |   5 +

 48 files changed, 2121 insertions(+), 102 deletions(-)

 create mode 100644 bin/tests/system/serve-stale/.gitignore

 create mode 100644 bin/tests/system/serve-stale/ans2/ans.pl.in

 create mode 100644 bin/tests/system/serve-stale/clean.sh

 create mode 100644 bin/tests/system/serve-stale/ns1/named1.conf.in

 create mode 100644 bin/tests/system/serve-stale/ns1/named2.conf.in

 create mode 100644 bin/tests/system/serve-stale/ns1/root.db

 create mode 100644 bin/tests/system/serve-stale/ns3/named.conf.in

 create mode 100644 bin/tests/system/serve-stale/prereq.sh

 create mode 100644 bin/tests/system/serve-stale/setup.sh

 create mode 100755 bin/tests/system/serve-stale/tests.sh

diff --git a/bin/named/config.c b/bin/named/config.c

index 63da4b0..b598f9b 100644

--- a/bin/named/config.c

+++ b/bin/named/config.c

@@ -182,13 +182,14 @@ options {\n\

 #ifdef HAVE_LMDB

 "	lmdb-mapsize 32M;\n"

 #endif

-"	max-acache-size 16M;\n\

-	max-cache-size 90%;\n\

+"	max-cache-size 90%;\n\

+	max-acache-size 16M;\n\

 	max-cache-ttl 604800; /* 1 week */\n\

 	max-clients-per-query 100;\n\

 	max-ncache-ttl 10800; /* 3 hours */\n\

 	max-recursion-depth 7;\n\

 	max-recursion-queries 75;\n\

+	max-stale-ttl 604800; /* 1 week */\n\

 	message-compression yes;\n\

 #	min-roots <obsolete>;\n\

 	minimal-any false;\n\

@@ -203,10 +204,14 @@ options {\n\

 	request-expire true;\n\

 	request-ixfr true;\n\

 	require-server-cookie no;\n\

+	resolver-nonbackoff-tries 3;\n\

+	resolver-retry-interval 800; /* in milliseconds */\n\

 #	rfc2308-type1 <obsolete>;\n\

 	root-key-sentinel yes;\n\

 	servfail-ttl 1;\n\

 #	sortlist <none>\n\

+	stale-answer-enable false;\n\

+	stale-answer-ttl 1; /* 1 second */\n\

 #	topology <none>\n\

 	transfer-format many-answers;\n\

 	v6-bias 50;\n\

diff --git a/bin/named/control.c b/bin/named/control.c

index df23c26..8b79850 100644

--- a/bin/named/control.c

+++ b/bin/named/control.c

@@ -282,6 +282,8 @@ ns_control_docommand(isccc_sexpr_t *message, bool readonly,

 		result = ns_server_validation(ns_g_server, lex, text);

 	} else if (command_compare(command, NS_COMMAND_ZONESTATUS)) {

 		result = ns_server_zonestatus(ns_g_server, lex, text);

+	} else if (command_compare(command, NS_COMMAND_SERVESTALE)) {

+		result = ns_server_servestale(ns_g_server, lex, text);

 	} else {

 		isc_log_write(ns_g_lctx, NS_LOGCATEGORY_GENERAL,

 			      NS_LOGMODULE_CONTROL, ISC_LOG_WARNING,

diff --git a/bin/named/include/named/control.h b/bin/named/include/named/control.h

index 8705fdd..1634154 100644

--- a/bin/named/include/named/control.h

+++ b/bin/named/include/named/control.h

@@ -69,6 +69,7 @@

 #define NS_COMMAND_MKEYS	"managed-keys"

 #define NS_COMMAND_DNSTAPREOPEN	"dnstap-reopen"

 #define NS_COMMAND_DNSTAP	"dnstap"

+#define NS_COMMAND_SERVESTALE	"serve-stale"

 isc_result_t

 ns_controls_create(ns_server_t *server, ns_controls_t **ctrlsp);

diff --git a/bin/named/include/named/log.h b/bin/named/include/named/log.h

index 56bfcd4..cd8db60 100644

--- a/bin/named/include/named/log.h

+++ b/bin/named/include/named/log.h

@@ -32,6 +32,7 @@

 #define NS_LOGCATEGORY_UPDATE_SECURITY	(&ns_g_categories[6])

 #define NS_LOGCATEGORY_QUERY_ERRORS	(&ns_g_categories[7])

 #define NS_LOGCATEGORY_TAT		(&ns_g_categories[8])

+#define NS_LOGCATEGORY_SERVE_STALE	(&ns_g_categories[9])

 /*

  * Backwards compatibility.

diff --git a/bin/named/include/named/query.h b/bin/named/include/named/query.h

index 9661f56..445b578 100644

--- a/bin/named/include/named/query.h

+++ b/bin/named/include/named/query.h

@@ -35,6 +35,18 @@ typedef struct ns_dbversion {

 	ISC_LINK(struct ns_dbversion)	link;

 } ns_dbversion_t;

+/*%

+ * nameserver recursion parameters, to uniquely identify a recursion

+ * query; this is used to detect a recursion loop

+ */

+typedef struct ns_query_recparam {

+	dns_rdatatype_t			qtype;

+	dns_name_t *			qname;

+	dns_fixedname_t			fqname;

+	dns_name_t *			qdomain;

+	dns_fixedname_t			fqdomain;

+} ns_query_recparam_t;

+

 /*% nameserver query structure */

 struct ns_query {

 	unsigned int			attributes;

@@ -63,6 +75,7 @@ struct ns_query {

 	unsigned int			dns64_aaaaoklen;

 	unsigned int			dns64_options;

 	unsigned int			dns64_ttl;

+

 	struct {

 		dns_db_t *      	db;

 		dns_zone_t *      	zone;

@@ -76,6 +89,8 @@ struct ns_query {

 		bool		authoritative;

 		bool		is_zone;

 	} redirect;

+

+	ns_query_recparam_t		recparam;

 	dns_keytag_t root_key_sentinel_keyid;

 	bool root_key_sentinel_is_ta;

 	bool root_key_sentinel_not_ta;

diff --git a/bin/named/include/named/server.h b/bin/named/include/named/server.h

index 8982d26..919ac28 100644

--- a/bin/named/include/named/server.h

+++ b/bin/named/include/named/server.h

@@ -224,7 +224,10 @@ enum {

 	dns_nsstatscounter_tcphighwater = 57,

-	dns_nsstatscounter_max = 58

+	dns_nsstatscounter_trystale = 58,

+	dns_nsstatscounter_usedstale = 59,

+

+	dns_nsstatscounter_max = 60

 };

 /*%

@@ -763,4 +766,12 @@ ns_server_mkeys(ns_server_t *server, isc_lex_t *lex, isc_buffer_t **text);

 isc_result_t

 ns_server_dnstap(ns_server_t *server, isc_lex_t *lex, isc_buffer_t **text);

+

+/*%

+ * Control whether stale answers are served or not when configured in

+ * named.conf.

+ */

+isc_result_t

+ns_server_servestale(ns_server_t *server, isc_lex_t *lex,

+		     isc_buffer_t **text);

 #endif /* NAMED_SERVER_H */

diff --git a/bin/named/log.c b/bin/named/log.c

index 3aa25e9..12f178b 100644

--- a/bin/named/log.c

+++ b/bin/named/log.c

@@ -38,6 +38,7 @@ static isc_logcategory_t categories[] = {

 	{ "update-security",		0 },

 	{ "query-errors",		0 },

 	{ "trust-anchor-telemetry",	0 },

+	{ "serve-stale",                0 },

 	{ NULL, 			0 }

 };

diff --git a/bin/named/query.c b/bin/named/query.c

index 0940714..882d69c 100644

--- a/bin/named/query.c

+++ b/bin/named/query.c

@@ -125,10 +125,14 @@

 #define REDIRECT(c)		(((c)->query.attributes & \

 				  NS_QUERYATTR_REDIRECT) != 0)

-/*% No QNAME Proof? */

+/*% Does the rdataset 'r' have an attached 'No QNAME Proof'? */

 #define NOQNAME(r)		(((r)->attributes & \

 				  DNS_RDATASETATTR_NOQNAME) != 0)

+/*% Does the rdataset 'r' contain a stale answer? */

+#define STALE(r)		(((r)->attributes & \

+				  DNS_RDATASETATTR_STALE) != 0)

+

 #ifdef WANT_QUERYTRACE

 static inline void

 client_trace(ns_client_t *client, int level, const char *message) {

@@ -217,6 +221,10 @@ static bool

 rpz_ck_dnssec(ns_client_t *client, isc_result_t qresult,

 	      dns_rdataset_t *rdataset, dns_rdataset_t *sigrdataset);

+static void

+recparam_update(ns_query_recparam_t *param, dns_rdatatype_t qtype,

+		const dns_name_t *qname, const dns_name_t *qdomain);

+

 /*%

  * Increment query statistics counters.

  */

@@ -470,6 +478,7 @@ query_reset(ns_client_t *client, bool everything) {

 	client->query.isreferral = false;

 	client->query.dns64_options = 0;

 	client->query.dns64_ttl = UINT32_MAX;

+	recparam_update(&client->query.recparam, 0, NULL, NULL);

 	client->query.root_key_sentinel_keyid = 0;

 	client->query.root_key_sentinel_is_ta = false;

 	client->query.root_key_sentinel_not_ta = false;

@@ -4254,6 +4263,54 @@ query_prefetch(ns_client_t *client, dns_name_t *qname,

 	dns_rdataset_clearprefetch(rdataset);

 }

+/*%

+ * Check whether the recursion parameters in 'param' match the current query's

+ * recursion parameters provided in 'qtype', 'qname', and 'qdomain'.

+ */

+static bool

+recparam_match(const ns_query_recparam_t *param, dns_rdatatype_t qtype,

+	       const dns_name_t *qname, const dns_name_t *qdomain)

+{

+	REQUIRE(param != NULL);

+

+	return (param->qtype == qtype &&

+	        param->qname != NULL && qname != NULL &&

+	        param->qdomain != NULL && qdomain != NULL &&

+	        dns_name_equal(param->qname, qname) &&

+	        dns_name_equal(param->qdomain, qdomain));

+}

+

+/*%

+ * Update 'param' with current query's recursion parameters provided in

+ * 'qtype', 'qname', and 'qdomain'.

+ */

+static void

+recparam_update(ns_query_recparam_t *param, dns_rdatatype_t qtype,

+		const dns_name_t *qname, const dns_name_t *qdomain)

+{

+	isc_result_t result;

+

+	REQUIRE(param != NULL);

+

+	param->qtype = qtype;

+

+	if (qname == NULL) {

+		param->qname = NULL;

+	} else {

+		param->qname = dns_fixedname_initname(&param->fqname);

+		result = dns_name_copy(qname, param->qname, NULL);

+		RUNTIME_CHECK(result == ISC_R_SUCCESS);

+	}

+

+	if (qdomain == NULL) {

+		param->qdomain = NULL;

+	} else {

+		param->qdomain = dns_fixedname_initname(&param->fqdomain);

+		result = dns_name_copy(qdomain, param->qdomain, NULL);

+		RUNTIME_CHECK(result == ISC_R_SUCCESS);

+	}

+}

+

 static isc_result_t

 query_recurse(ns_client_t *client, dns_rdatatype_t qtype, dns_name_t *qname,

 	      dns_name_t *qdomain, dns_rdataset_t *nameservers,

@@ -4263,6 +4320,19 @@ query_recurse(ns_client_t *client, dns_rdatatype_t qtype, dns_name_t *qname,

 	dns_rdataset_t *rdataset, *sigrdataset;

 	isc_sockaddr_t *peeraddr;

+	/*

+	 * Check recursion parameters from the previous query to see if they

+	 * match.  If not, update recursion parameters and proceed.

+	 */

+	if (recparam_match(&client->query.recparam, qtype, qname, qdomain)) {

+		ns_client_log(client, NS_LOGCATEGORY_CLIENT,

+			      NS_LOGMODULE_QUERY, ISC_LOG_INFO,

+			      "recursion loop detected");

+		return (ISC_R_FAILURE);

+	}

+

+	recparam_update(&client->query.recparam, qtype, qname, qdomain);

+

 	if (!resuming)

 		inc_stats(client, dns_nsstatscounter_recursion);

@@ -6780,6 +6850,7 @@ query_find(ns_client_t *client, dns_fetchevent_t *event, dns_rdatatype_t qtype)

 	int line = -1;

 	bool dns64_exclude, dns64, rpz;

 	bool nxrewrite = false;

+	bool want_stale = false;

 	bool redirected = false;

 	dns_clientinfomethods_t cm;

 	dns_clientinfo_t ci;

@@ -7089,6 +7160,7 @@ query_find(ns_client_t *client, dns_fetchevent_t *event, dns_rdatatype_t qtype)

 		type = qtype;

  restart:

+	// query_start

 	CTRACE(ISC_LOG_DEBUG(3), "query_find: restart");

 	want_restart = false;

 	authoritative = false;

@@ -7233,6 +7305,7 @@ query_find(ns_client_t *client, dns_fetchevent_t *event, dns_rdatatype_t qtype)

 	}

  db_find:

+	// query_lookup

 	CTRACE(ISC_LOG_DEBUG(3), "query_find: db_find");

 	/*

 	 * We'll need some resources...

@@ -7290,6 +7363,35 @@ query_find(ns_client_t *client, dns_fetchevent_t *event, dns_rdatatype_t qtype)

 	if (!is_zone)

 		dns_cache_updatestats(client->view->cache, result);

+	if (want_stale) {

+		char namebuf[DNS_NAME_FORMATSIZE];

+		bool success;

+

+		client->query.dboptions &= ~DNS_DBFIND_STALEOK;

+		want_stale = false;

+

+		if (dns_rdataset_isassociated(rdataset) &&

+		    dns_rdataset_count(rdataset) > 0 &&

+		    STALE(rdataset)) {

+			rdataset->ttl = client->view->staleanswerttl;

+			success = true;

+		} else {

+			success = false;

+		}

+

+		dns_name_format(client->query.qname,

+				namebuf, sizeof(namebuf));

+		isc_log_write(ns_g_lctx, NS_LOGCATEGORY_SERVE_STALE,

+			      NS_LOGMODULE_QUERY, ISC_LOG_INFO,

+			      "%s resolver failure, stale answer %s",

+			      namebuf, success ? "used" : "unavailable");

+

+		if (!success) {

+			QUERY_ERROR(DNS_R_SERVFAIL);

+			goto cleanup;

+		}

+	}

+

  resume:

 	CTRACE(ISC_LOG_DEBUG(3), "query_find: resume");

@@ -7635,6 +7737,7 @@ query_find(ns_client_t *client, dns_fetchevent_t *event, dns_rdatatype_t qtype)

 		 * The cache doesn't even have the root NS.  Get them from

 		 * the hints DB.

 		 */

+		// query_notfound

 		INSIST(!is_zone);

 		if (db != NULL)

 			dns_db_detach(&db);

@@ -7697,12 +7800,14 @@ query_find(ns_client_t *client, dns_fetchevent_t *event, dns_rdatatype_t qtype)

 		 */

 		/* FALLTHROUGH */

 	case DNS_R_DELEGATION:

+		// query_delegation

 		authoritative = false;

 		if (is_zone) {

 			/*

 			 * Look to see if we are authoritative for the

 			 * child zone if the query type is DS.

 			 */

+			// query_zone_delegation

 			if (!RECURSIONOK(client) &&

 			    (options & DNS_GETDB_NOEXACT) != 0 &&

 			    qtype == dns_rdatatype_ds) {

@@ -8089,6 +8194,7 @@ query_find(ns_client_t *client, dns_fetchevent_t *event, dns_rdatatype_t qtype)

 						       false, true);

 			}

 		}

+		// query_nxdomain

 		if (dns_rdataset_isassociated(rdataset)) {

 			/*

 			 * If we've got a NSEC record, we need to save the

@@ -8409,7 +8515,8 @@ query_find(ns_client_t *client, dns_fetchevent_t *event, dns_rdatatype_t qtype)

 		/*

 		 * If we have a zero ttl from the cache refetch it.

 		 */

-		if (!is_zone && !resuming && rdataset->ttl == 0 &&

+		// query_cname

+		if (!is_zone && !resuming && !STALE(rdataset) && rdataset->ttl == 0 &&

 		    RECURSIONOK(client))

 		{

 			if (dns_rdataset_isassociated(rdataset))

@@ -8627,7 +8734,11 @@ query_find(ns_client_t *client, dns_fetchevent_t *event, dns_rdatatype_t qtype)

 			 "query_find: unexpected error after resuming: %s",

 			 isc_result_totext(result));

 		CTRACE(ISC_LOG_ERROR, errmsg);

-		QUERY_ERROR(DNS_R_SERVFAIL);

+		if (resuming) {

+			want_stale = true;

+		} else {

+			QUERY_ERROR(DNS_R_SERVFAIL);

+		}

 		goto cleanup;

 	}

@@ -8883,7 +8994,7 @@ query_find(ns_client_t *client, dns_fetchevent_t *event, dns_rdatatype_t qtype)

 		/*

 		 * If we have a zero ttl from the cache refetch it.

 		 */

-		if (!is_zone && !resuming && rdataset->ttl == 0 &&

+		if (!is_zone && !resuming && !STALE(rdataset) && rdataset->ttl == 0 &&

 		    RECURSIONOK(client))

 		{

 			if (dns_rdataset_isassociated(rdataset))

@@ -8894,6 +9005,7 @@ query_find(ns_client_t *client, dns_fetchevent_t *event, dns_rdatatype_t qtype)

 			if (node != NULL)

 				dns_db_detachnode(db, &node);

+			// query_respond

 			INSIST(!REDIRECT(client));

 			result = query_recurse(client, qtype,

 					       client->query.qname,

@@ -9174,6 +9286,7 @@ query_find(ns_client_t *client, dns_fetchevent_t *event, dns_rdatatype_t qtype)

 				       dns_fixedname_name(&wildcardname),

 				       true, false);

  cleanup:

+	// query_done

 	CTRACE(ISC_LOG_DEBUG(3), "query_find: cleanup");

 	/*

 	 * General cleanup.

@@ -9230,6 +9343,49 @@ query_find(ns_client_t *client, dns_fetchevent_t *event, dns_rdatatype_t qtype)

 		goto restart;

 	}

+	if (want_stale) {

+		dns_ttl_t stale_ttl = 0;

+		isc_result_t result;

+		bool staleanswersok = false;

+

+		/*

+		 * Stale answers only make sense if stale_ttl > 0 but

+		 * we want rndc to be able to control returning stale

+		 * answers if they are configured.

+		 */

+		dns_db_attach(client->view->cachedb, &db);

+		result = dns_db_getservestalettl(db, &stale_ttl);

+		if (result == ISC_R_SUCCESS && stale_ttl > 0)  {

+			switch (client->view->staleanswersok) {

+			case dns_stale_answer_yes:

+				staleanswersok = true;

+				break;

+			case dns_stale_answer_conf:

+				staleanswersok =

+					client->view->staleanswersenable;

+				break;

+			case dns_stale_answer_no:

+				staleanswersok = false;

+				break;

+			}

+		} else {

+			staleanswersok = false;

+		}

+

+		if (staleanswersok) {

+			client->query.dboptions |= DNS_DBFIND_STALEOK;