Introducing --overwrite option

Sometimes it is needed to overwrite existing output files.
This has been requested in the past (#927). For safety reasons
it was not implemented.

However I realized that it could be useful. It requires some
responsible usage though.

Code added, help() and manpages added -- warnings added too.

Author: drwetter

doc/testssl.1

@@ -496,7 +496,10 @@ whole 9 yards
 \fB\-\-severity <severity>\fR For CSV and both JSON outputs this will only add findings to the output file if a severity is equal or higher than the \fBseverity\fR value specified\. Allowed are \fB<LOW|MEDIUM|HIGH|CRITICAL>\fR\. WARN is another level which translates to a client\-side scanning error or problem\. Thus you will always see them in a file if they occur\.
 .
 .P
-\fB\-\-append\fR Normally, if an output file already exists and it has a file size greater zero, testssl\.sh will prompt you to manually remove the file exit with an error\. \fB\-\-append\fR however will append to this file, without a header\. The environment variable APPEND does the same\. Be careful using this switch/variable\. A complementary option which overwrites an existing file doesn\'t exist per design\.
+\fB\-\-append\fR Normally, if an output file already exists and it has a file size greater zero, testssl\.sh will prompt you to manually remove the file and then exit with an error\. \fB\-\-append\fR however will append to this file, without a header\. The environment variable APPEND does the same\. Be careful using this switch/variable\.
+.
+.P
+\fB\-\-overwrite\fR Normally, if an output file already exists and it has a file size greater zero, testssl\.sh will not allow you to overwrite this file\. This option will do that \fBwithout any warning\fR\. The environment variable OVERWRITE does the same\. Be careful, you have been warned!
 .
 .P
 \fB\-\-outprefix <fname_prefix>\fR Prepend output filename prefix \fIfname_prefix\fR before \'${NODE}\-\'\. You can use as well the environment variable FNAME_PREFIX\. Using this any output files will be named \fB<fname_prefix>\-${NODE}\-p${port}${YYYYMMDD\-HHMM}\.<format>\fR when no file name of the respective output option was specified\. If you do not like the separator \'\-\' you can as well supply a \fB<fname_prefix>\fR ending in \'\.\', \'_\' or \',\'\. In this case or if you already supplied \'\-\' no additional \'\-\' will be appended to \fB<fname_prefix>\fR\.

doc/testssl.1.html

@@ -327,7 +327,9 @@ Rating automatically gets disabled, to not give a wrong or misleading grade, whe
 
 `--severity <severity>` For CSV and both JSON outputs this will only add findings to the output file if a severity is equal or higher than the `severity` value specified. Allowed are `<LOW|MEDIUM|HIGH|CRITICAL>`. WARN is another level which translates to a client-side scanning error or problem. Thus you will always see them in a file if they occur.
 
-`--append` Normally, if an output file already exists and it has a file size greater zero, testssl.sh will prompt you to manually remove the file exit with an error. `--append` however will append to this file, without a header. The environment variable APPEND does the same. Be careful using this switch/variable. A complementary option which overwrites an existing file doesn't exist per design.
+`--append` Normally, if an output file already exists and it has a file size greater zero, testssl.sh will prompt you to manually remove the file and exit with an error. `--append` however will append to this file, without a header. The environment variable APPEND does the same. Be careful using this switch/variable. A complementary option which overwrites an existing file doesn't exist per design.
+
+`--overwrite` Normally, if an output file already exists and it has a file size greater zero, testssl.sh will not allow you to overwrite this file. This option will do that **without any warning**. The environment variable OVERWRITE does the same. Be careful, you have been warned!
 
 `--outprefix <fname_prefix>` Prepend output filename prefix <fname_prefix> before '${NODE}-'. You can use as well the environment variable FNAME_PREFIX. Using this any output files will be named `<fname_prefix>-${NODE}-p${port}${YYYYMMDD-HHMM}.<format>` when no file name of the respective output option was specified. If you do not like the separator '-' you can as well supply a `<fname_prefix>` ending in '.',  '_' or ','. In this case or if you already supplied '-' no additional '-' will be appended to `<fname_prefix>`.
 

doc/testssl.1.md

@@ -179,7 +179,8 @@ CSVFILE="${CSVFILE:-""}"                # csvfile if used
 HTMLFILE="${HTMLFILE:-""}"              # HTML if used
 FNAME=${FNAME:-""}                      # file name to read commands from
 FNAME_PREFIX=${FNAME_PREFIX:-""}        # output filename prefix, see --outprefix
-APPEND=${APPEND:-false}                 # append to csv/json file instead of overwriting it
+APPEND=${APPEND:-false}                 # append to csv/json/html/log file
+OVERWRITE=${OVERWRITE:-false}           # overwriting csv/json/html/log file
 [[ -z "$NODNS" ]] && declare NODNS      # If unset it does all DNS lookups per default. "min" only for hosts or "none" at all
 HAS_IPv6=${HAS_IPv6:-false}             # if you have OpenSSL with IPv6 support AND IPv6 networking set it to yes
 ALL_CLIENTS=${ALL_CLIENTS:-false}       # do you want to run all client simulation form all clients supplied by SSLlabs?
@@ -1349,7 +1350,10 @@ json_header() {
      if "$APPEND"; then
           JSONHEADER=false
      else
-          [[ -s "$JSONFILE" ]] && fatal "non-empty \"$JSONFILE\" exists. Either use \"--append\" or (re)move it" $ERR_FCREATE
+          if [[ -s "$JSONFILE" ]]; then
+               "$OVERWRITE" || fatal "non-empty \"$JSONFILE\" exists. Either use \"--append\" or (re)move it" $ERR_FCREATE
+               cp /dev/null "$JSONFILE"
+          fi
           "$do_json" && echo "[" > "$JSONFILE"
           "$do_pretty_json" && echo "{" > "$JSONFILE"
      fi
@@ -1390,7 +1394,10 @@ csv_header() {
      if "$APPEND"; then
           CSVHEADER=false
      else
-          [[ -s "$CSVFILE" ]] && fatal "non-empty \"$CSVFILE\" exists. Either use \"--append\" or (re)move it" $ERR_FCREATE
+          if [[ -s "$CSVFILE" ]]; then
+               "$OVERWRITE" || fatal "non-empty \"$CSVFILE\" exists. Either use \"--append\" or (re)move it" $ERR_FCREATE
+               cp /dev/null "$CSVFILE"
+          fi
           touch "$CSVFILE"
           if "$GIVE_HINTS"; then
                fileout_csv_finding "id" "fqdn/ip" "port" "severity" "finding" "cve" "cwe" "hint"
@@ -1440,7 +1447,10 @@ html_header() {
      if "$APPEND"; then
           HTMLHEADER=false
      else
-          [[ -s "$HTMLFILE" ]] && fatal "non-empty \"$HTMLFILE\" exists. Either use \"--append\" or (re)move it" $ERR_FCREATE
+          if [[ -s "$HTMLFILE" ]]; then
+               "$OVERWRITE" || fatal "non-empty \"$HTMLFILE\" exists. Either use \"--append\" or (re)move it" $ERR_FCREATE
+               cp /dev/null "$HTMLFILE"
+          fi
           html_out "<?xml version=\"1.0\" encoding=\"UTF-8\" ?>\n"
           html_out "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\" \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\">\n"
           html_out "<!-- This file was created with testssl.sh. https://testssl.sh -->\n"
@@ -1499,7 +1509,10 @@ prepare_logging() {
      fi
 
      if ! "$APPEND"; then
-          [[ -s "$LOGFILE" ]] && fatal "non-empty \"$LOGFILE\" exists. Either use \"--append\" or (re)move it" $ERR_FCREATE
+          if [[ -s "$LOGFILE" ]]; then
+               "$OVERWRITE" || fatal "non-empty \"$LOGFILE\" exists. Either use \"--append\" or (re)move it" $ERR_FCREATE
+               cp /dev/null "$LOGFILE"
+          fi
      fi
      tmln_out "## Scan started as: \"$PROG_NAME $CMDLINE\"" >>"$LOGFILE"
      tmln_out "## at $HNAME:$OPENSSL_LOCATION" >>"$LOGFILE"
@@ -19260,6 +19273,7 @@ file output options (can also be preset via environment variables)
      --hints                       additional hints to findings
      --severity <severity>         severities with lower level will be filtered for CSV+JSON, possible values <LOW|MEDIUM|HIGH|CRITICAL>
      --append                      if (non-empty) <logfile>, <csvfile>, <jsonfile> or <htmlfile> exists, append to file. Omits any header
+     --overwrite                   if <logfile>, <csvfile>, <jsonfile> or <htmlfile> exists it overwrites it without any warning
      --outprefix <fname_prefix>    before  '\${NODE}.' above prepend <fname_prefix>
 
 
@@ -22118,7 +22132,12 @@ parse_cmd_line() {
                     do_csv=true
                     do_logging=true
                     ;;
+               --overwrite)
+                    "$APPEND" && fatal "using --overwrite and --append is contradicting" $ERR_CMDLINE
+                    OVERWRITE=true
+                    ;;
                --append)
+                    "$OVERWRITE" && fatal "using --append and --overwrite is contradicting" $ERR_CMDLINE
                     APPEND=true
                     ;;
                --outprefix)