tools/xenconsoled: Increase file descriptor limit

XenServer's VM density testing uncovered a regression when moving from
sysvinit to systemd where the file descriptor limit dropped from 4096 to
1024. (XenServer had previously inserted a ulimit statement into its
initscripts.)

One solution is to use LimitNOFILE=4096 in xenconsoled.service to match the
lost ulimit, but that is only a stopgap solution.

As Xenconsoled genuinely needs a large number of file descriptors if a large
number of domains are running, attempt to increase the limit.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Acked-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
(cherry picked from commit 588df84c0d)

tools/console/daemon/main.c

@@ -26,6 +26,7 @@
 #include <string.h>
 #include <signal.h>
 #include <sys/types.h>
+#include <sys/resource.h>
 
 #include "xenctrl.h"
 
@@ -55,6 +56,41 @@ static void version(char *name)
 	printf("Xen Console Daemon 3.0\n");
 }
 
+static void increase_fd_limit(void)
+{
+	/*
+	 * We require many file descriptors:
+	 * - per domain: pty master, pty slave, logfile and evtchn
+	 * - misc extra: hypervisor log, privcmd, gntdev, std...
+	 *
+	 * Allow a generous 1000 for misc, and calculate the maximum possible
+	 * number of fds which could be used.
+	 */
+	unsigned min_fds = (DOMID_FIRST_RESERVED * 4) + 1000;
+	struct rlimit lim, new = { min_fds, min_fds };
+
+	if (getrlimit(RLIMIT_NOFILE, &lim) < 0) {
+		fprintf(stderr, "Failed to obtain fd limit: %s\n",
+			strerror(errno));
+		exit(1);
+	}
+
+	/* Do we already have sufficient? Great! */
+	if (lim.rlim_cur >= min_fds)
+		return;
+
+	/* Try to increase our limit. */
+	if (setrlimit(RLIMIT_NOFILE, &new) < 0)
+		syslog(LOG_WARNING,
+		       "Unable to increase fd limit from {%llu, %llu} to "
+		       "{%llu, %llu}: (%s) - May run out with lots of domains",
+		       (unsigned long long)lim.rlim_cur,
+		       (unsigned long long)lim.rlim_max,
+		       (unsigned long long)new.rlim_cur,
+		       (unsigned long long)new.rlim_max,
+		       strerror(errno));
+}
+
 int main(int argc, char **argv)
 {
 	const char *sopts = "hVvit:o:";
@@ -154,6 +190,8 @@ int main(int argc, char **argv)
 	openlog("xenconsoled", syslog_option, LOG_DAEMON);
 	setlogmask(syslog_mask);
 
+	increase_fd_limit();
+
 	if (!is_interactive) {
 		daemonize(pidfile ? pidfile : "/var/run/xenconsoled.pid");
 	}