ci: Check and confirm Vim feature flags exist before testing

Vim tests for features such as python3 relies on checking the feature
flag exists by doing `has('python3')`. However, if the feature itself is
broken and the flag returns 0, the relevant tests will simply silently
get ignored and CI will passed erroneously. As a preventive measure, as
basic checks to make sure certain feature flags are correct as a basic
smoke test.

Currently only checking two types of feature flags:

1. Features that depend on system packages being installed properly
   (e.g. sodium) and could be erroneously dropped if the CI environment
   changed or a bug exists in the configure script.
2. Scripting languages. When in dynamic mode, these feature flags (e.g.
   "ruby", "python3") will return 0 when the lib cannot be found or the
   code has an initialization bug. This happened in #16964 where CI
   still passed despite Python 3 being broken.

closes: #16998

Signed-off-by: Yee Cheng Chin <ychin.git@gmail.com>
Signed-off-by: Christian Brabandt <cb@256bit.org>

.github/workflows/ci.yml

@@ -266,6 +266,12 @@ jobs:
           "${SRCDIR}"/vim --version
           "${SRCDIR}"/vim -u NONE -i NONE --not-a-term -esNX -V1 -S ci/if_ver-1.vim -c quit
           "${SRCDIR}"/vim -u NONE -i NONE --not-a-term -esNX -V1 -S ci/if_ver-2.vim -c quit
+          if ${{ matrix.features == 'huge' }}; then
+            # Also check that optional and dynamic features are configured and working
+            "${SRCDIR}"/vim -u NONE -i NONE --not-a-term -esNX -V1 \
+              -c "let g:required=['gettext', 'sodium', 'sound', 'perl', 'python3', 'lua', 'ruby', 'tcl']" \
+              -S ci/if_feat_check.vim -c quit
+          fi
 
       - name: Test
         if: matrix.architecture != 'arm64'
@@ -392,6 +398,12 @@ jobs:
           "${SRCDIR}"/vim --version
           "${SRCDIR}"/vim -u NONE -i NONE --not-a-term -esNX -V1 -S ci/if_ver-1.vim -c quit
           "${SRCDIR}"/vim -u NONE -i NONE --not-a-term -esNX -V1 -S ci/if_ver-2.vim -c quit
+          if ${{ matrix.features == 'huge' }}; then
+            # Also check that optional and dynamic features are configured and working
+            "${SRCDIR}"/vim -u NONE -i NONE --not-a-term -esNX -V1 \
+              -c "let g:required=['sound', 'perl', 'python3', 'lua', 'ruby', 'tcl']" \
+              -S ci/if_feat_check.vim -c quit
+          fi
 
       - name: Install packages for testing
         run: |
@@ -663,6 +675,11 @@ jobs:
             src\vim --version || exit 1
             src\vim -u NONE -i NONE --not-a-term -esNX -V1 -S ci/if_ver-1.vim -c quit
             src\vim -u NONE -i NONE --not-a-term -esNX -V1 -S ci/if_ver-2.vim -c quit
+            if "${{ matrix.features }}"=="HUGE" (
+              src\vim -u NONE -i NONE --not-a-term -esNX -V1 ^
+                -c "let g:required=['gettext', 'sodium', 'sound', 'python3', 'lua']" ^
+                -S ci/if_feat_check.vim -c quit
+            )
           )
 
       #- name: Prepare Artifact

Filelist

@@ -23,6 +23,7 @@ SRC_ALL =	\
 		ci/appveyor.bat \
 		ci/config.mk*.sed \
 		ci/if_ver*.vim \
+		ci/if_feat_check.vim \
 		ci/setup-xvfb.sh \
 		ci/remove_snap.sh \
 		src/Make_all.mak \

ci/if_feat_check.vim

@@ -0,0 +1,15 @@
+if 1 " This prevents it from being run in tiny versions
+  " Check for required features
+  if exists("g:required")
+    for feature in g:required
+      if !has(feature)
+        echo "Error: Feature '" .. feature .. "' not found"
+        echo ''
+        cquit
+      endif
+    endfor
+    echo "\nChecked features: " .. string(g:required)
+    echo ''
+  endif
+endif
+" vim: sts=2 sw=2 et

ci/if_ver-1.vim

@@ -1,6 +1,6 @@
-" Print all interface versions for Ubuntu. Part 1.
+" Print all interface versions. Part 1.
 
-if 1
+if 1 " This prevents it from being run in tiny versions
   execute 'source' expand('<sfile>:h') .. '/if_ver-cmd.vim'
 
   echo "*** Interface versions ***\n"

ci/if_ver-2.vim

@@ -1,6 +1,8 @@
-" Print py3 interface versions for Ubuntu. Part 2.
+" Print py3 interface versions. Part 2.
+" This is done separately from part 1 because Vim cannot concurrently load
+" Python 2 and 3 together.
 
-if 1
+if 1 " This prevents it from being run in tiny versions
   execute 'source' expand('<sfile>:h') .. '/if_ver-cmd.vim'
 
   echo 'Python 3:'