HowTo: Install Python, PIP and cx_Oracle on MSYS2 / Windows

Here’s how to install Python, PIP package installer and cx_Oracle extension module on MSYS2:

$ pacman -S python
warning: python-3.8.3-1 is up to date — reinstalling
resolving dependencies…
looking for conflicting packages…

Packages (1) python-3.8.3-1

Total Installed Size:  108.66 MiB
Net Upgrade Size:        0.00 MiB

:: Proceed with installation? [Y/n] y

…

$ pacman -S cx_Oracle
error: target not found: cx_Oracle

$ pip install cx_Oracle
bash: pip: command not found

$ pacman -S python-pip
resolving dependencies…
looking for conflicting packages…

Packages (2) python-setuptools-47.1.1-1  python-pip-20.1.1-1

Total Download Size:    2.22 MiB
Total Installed Size:  10.91 MiB

:: Proceed with installation? [Y/n] y
:: Retrieving packages…
…

$ pip install cx_Oracle
Collecting cx_Oracle
  Downloading cx_Oracle-8.0.0.tar.gz (325 kB)
     |████████████████████████████████| 325 kB 693 kB/s
Using legacy setup.py install for cx-Oracle, since package ‘wheel’ is not installed.
Installing collected packages: cx-Oracle
    Running setup.py install for cx-Oracle … error
    ERROR: Command errored out with exit status 1:
     command: /usr/bin/python3.exe -u -c ‘import sys, setuptools, tokenize; sys.argv[0] = ‘"’"’/tmp/pip-install-wqtzv2rv/cx-Oracle/setup.py’"’"’; __file__=’"’"’/tmp/pip-install-wqtzv2rv/cx-Oracle/setup.py’"’"’;f=getattr(tokenize, ‘"’"’open’"’"’, open)(__file__);code=f.read().replace(‘"’"’\r\n’"’"’, ‘"’"’\n’"’"’);f.close();exec(compile(code, __file__, ‘"’"’exec’"’"’))’ install –record /tmp/pip-record-mui2mjs9/install-record.txt –single-version-externally-managed –compile –install-headers /usr/include/python3.8/cx-Oracle
         cwd: /tmp/pip-install-wqtzv2rv/cx-Oracle/
    Complete output (17 lines):
    running install
    running build
    running build_ext
    building ‘cx_Oracle’ extension
    creating build
    creating build/temp.msys-3.1.5-x86_64-3.8
    creating build/temp.msys-3.1.5-x86_64-3.8/src
    creating build/temp.msys-3.1.5-x86_64-3.8/odpi
    creating build/temp.msys-3.1.5-x86_64-3.8/odpi/src
    x86_64-pc-msys-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O2 -pipe -march=x86-64 -mtune=generic -O2 -pipe -march=x86-64 -mtune=generic -O2 -pipe -DCXO_BUILD_VERSION=8.0.0 -Iodpi/include -Iodpi/src -I/usr/include/python3.8 -c src/cxoApiType.c -o build/temp.msys-3.1.5-x86_64-3.8/src/cxoApiType.o
    In file included from src/cxoModule.h:14,
                     from src/cxoApiType.c:11:
    /usr/include/python3.8/Python.h:44:10: fatal error: crypt.h: No such file or directory
       44 | #include <crypt.h>
          |          ^~~~~~~~~
    compilation terminated.
    error: command ‘x86_64-pc-msys-gcc’ failed with exit status 1
    —————————————-
ERROR: Command errored out with exit status 1: /usr/bin/python3.exe -u -c ‘import sys, setuptools, tokenize; sys.argv[0] = ‘"’"’/tmp/pip-install-wqtzv2rv/cx-Oracle/setup.py’"’"’; __file__=’"’"’/tmp/pip-install-wqtzv2rv/cx-Oracle/setup.py’"’"’;f=getattr(tokenize, ‘"’"’open’"’"’, open)(__file__);code=f.read().replace(‘"’"’\r\n’"’"’, ‘"’"’\n’"’"’);f.close();exec(compile(code, __file__, ‘"’"’exec’"’"’))’ install –record /tmp/pip-record-mui2mjs9/install-record.txt –single-version-externally-managed –compile –install-headers /usr/include/python3.8/cx-Oracle Check the logs for full command output.

Found a solution here for other Linuxes, mentioning

apt install python-dev clang libcrypt-dev

Via MSYS2 package manager (pacman), libcrypt and libcrypt-devel seem to be available. The second one should fetch the first one if needed, together with header files for development.

$ pacman -S libcrypt-devel
resolving dependencies…
looking for conflicting packages…

Packages (1) libcrypt-devel-2.1-2

Total Download Size:   0.04 MiB
Total Installed Size:  0.04 MiB

:: Proceed with installation? [Y/n] y
…

Now we can install cx_Oracle succesfully:

$ pip install cx_Oracle
Collecting cx_Oracle
  Using cached cx_Oracle-8.0.0.tar.gz (325 kB)
Using legacy setup.py install for cx-Oracle, since package ‘wheel’ is not installed.
Installing collected packages: cx-Oracle
    Running setup.py install for cx-Oracle … done
Successfully installed cx-Oracle-8.0.0

Using cross-platform fswatch utility for monitoring file system changes

I was just looking for a way to monitor for file system changs in a cross-platform way.

After searching the answers at a related question on StackExchange and the ones at the other question mentioned as possible duplicate in that question’s comments, I think I’d go with fswatch since it is has cross-platform support. Contributed the following answer to collect info I gathered on fswatch.

It uses different kinds of monitors for different OS and can choose the appropriate one automatically, or allow one to specify which one to use and even pass custom platform-specific parameters to the respective monitor.

The list of monitors it currently supports is:

· A monitor based on the File System Events API of Apple OS X.

· A monitor based on kqueue, an event notification interface introduced in FreeBSD 4.1 and supported on most *BSD systems (including OS X).

· A monitor based on inotify, a Linux kernel subsystem that reports file system changes to applications.

· A monitor based on File Events Notification, a Solaris/Illumos kernel API that reports file events.

· A monitor based on ReadDirectoryChangesW, a Microsoft Windows API that reports changes to a directory.

· A monitor which periodically stats the file system, saves file modification times in memory and manually calculates file system changes, which can work on any operating system where stat (2) can be used.

It seems to be available via apt-get on Debian/Ubuntu Linuxes. See how to install via apt-get and use fswatch.

FreeBSD and OS-X package-based installation support for fswatch is provided by its author.

Can also build and install it at other OSes, found an article+video showing how to make and install fswatch on CentOS.

There is also an other article that shows the same manual process to build/install (and use) fswatch for Linux.

Windows-based package-based installation support doesn’t seem to be available yet (e.g. no package for Chocolatey yet, and no package for Vcpkg)

There is also extensive documentation for fswatch, though docs for the latest 1.5 version point to 1.4 ones currently. See how to convert commands for fswatch 0.x to fswatch 1.x

Tips for manually choosing a monitor (currently not updated to mention all monitors)

Read about fswatch usage here, here and here and a tutorial intro here

A library named libfswatch is kept in sync with the fswatch tool. See here and a newer doc here. Note that the library is versioned differently from fswatch utility itself. Specifically, the 1.14.0 library doc states:

"libtool’s versioning scheme is described by three integers: current:revision:age.

· current is the most recent interface number implemented by the library.

· revision is the implementation number of the current interface.

· age is the difference between the newest and the oldest interface that the library implements.

Beware that there is also another similar s/w called fswatch (that is go-related I think).

Python rstrip and whitespace

Was just reading

https://www.w3schools.com/python/ref_string_rstrip.asp

which says

The rstrip() method removes any trailing characters (characters at the end a string), space is the default trailing character to remove.

Notes:

1) the chars param was added at Python 2.2.3, can’t use it at older versions as noted at

https://docs.python.org/2.6/library/string.html

string.rstrip(s[, chars])

Return a copy of the string with trailing characters removed. If chars is omitted or None, whitespace characters are removed. If given and not None, chars must be a string; the characters in the string will be stripped from the end of the string this method is called on.

Changed in version 2.2.3: The chars parameter was added. The chars parameter cannot be passed in earlier 2.2 versions.

2) from the official doc and other docs too I read the default is to remove whitespace characters, not just the space char:

https://python-reference.readthedocs.io/en/latest/docs/str/rstrip.html

chars

Optional. String specifying the set of characters to be removed. If omitted or None, the chars argument defaults to removing whitespace. The chars argument is not a prefix; rather, all combinations of its values are stripped.

Not sure what is considered whitespace in various Python versions though. At least in Python2 it wasn’t removing \r in both Solaris and MSYS2 (a POSIX environment on Windows) where I just tried.

For example, I was just debugging some program that was working in Python 3, but in Python 2 it was moving the cursor to the start of the line when printing a raw_input prompt with some string it had read before…
…the issue proved to be that it was opening a file with ‘r’ mode instead of ‘rU’ which is universal newlines mode – https://www.python.org/dev/peps/pep-0278/ – converts \r\n to \n – and it seems that rstrip was failing to remove the \r from the end of those strings.

In Python 3 it was either using the universal newlines mode by default and thus stripping the \r from strings while reading from the file, or the rstrip was removing \r too in Python3, but I guess it was the 1st case (didn’t do any more check since the universal newlines read file open mode fixed the issue in Python 2 for me).

Speaking of that, I wonder whether Python considers whitespace differently on Windows and on Unixes (aka having [l/r]strip commands remove \r on the 1st but not on the 2nd case), which would be an extra complexity when writing portable s/w.

HowTo: make raw_input & input work the same in both Python 2 and 3

Was just trying to make a Python 2 script work in Python 3 and at first it seemed I just needed a small change to add missing parentheses to the argument of a couple of print statements.

But then another issue came up, it wasn’t understanding the command raw_input(somePrompt) that was occuring at various places in that file to input text from the console.

Various solutions were proposed at

https://stackoverflow.com/questions/954834/how-do-i-use-raw-input-in-python-3/62825723

but I think I came up with a cleaner looking solution that works in both Python2 and Python3, that should allow one to use either raw_input or input in both Python2 and Python3 with the semantics of Python2’s raw_input (aka the semantics of Python3’s input).

# raw_input isn't defined in Python3.x, whereas input wasn't behaving 
# like raw_input in Python 2.x. This should make both input and raw_input 
# work in Python 2.x/3.x like the raw_input from Python 2.x 
try: input = raw_input
except NameError: raw_input = input

In practice this came up from ideas at other answers on that SO thread. It tries to define input as raw_input which should fail in Python3.x since raw_input is undefined. In that case it will catch a NameError exception and do the reverse, aka define raw_input as input. However in Python2 the first command should execute fine, overriding the insecure input of Python2.x to work the same as raw_input (not trying to interpret input strings that is). Actually that is what Python 3.x input does.

Wonder why they didn’t declare it like that in the first place though, breaking compilation of Python2 programs. After all semantically only Python2 programs that were using the old insecure input would have an issue, not those that were using raw_input which is the semantics Python3 promotes with its newer input.

HowTo: Use DISM and SFC tools to check+fix your Windows installation

If you’re having issues with your Windows 7 or newer, you should consider whether its installation has become corrupted (due to malicious software or hard drive errors).

After doing a disk check (say by right clicking the appropriate drive under my computer and selecting Properties, then Tools tab and Error checking) and a complete virus scan (on Win10 you can click the shield icon of Windows defender in the taskbar tray and at scanning options choose to do a full scan – or if you have installed some third-party antivirus double-click its icon in the taskbar tray and when its GUI opens up opt to do a full scan), then try the following steps to repair your Windows installation:

1. Press WIN+R to open Run dialog

2. Type in:

CMD

Hold down CTRL+SHIFT keys and click OK to open the command line window in Administrator mode (do press Yes at the User Account Control prompt)

A (usually) black text-based console window will open up and you’ll be greated with something like:

Microsoft Windows [Version 10.0.18363.720]
(c) 2019 Microsoft Corporation. All rights reserved.

and then a prompt like:

C:\Windows\system32>

3. Type in the following and press the ENTER key:

DISM.exe /Online /Cleanup-image /Scanhealth

and press ENTER to execute the DISM tool with the option to check the windows image health and wait patiently for it to complete

Deployment Image Servicing and Management tool
Version: 10.0.18362.1

Image Version: 10.0.18363.720

[==========================100.0%==========================] The component store is repairable.
The operation completed successfully.

4. In case you see a message that the component store is repairable, then when greeted with the C:\Windows\system32> prompt again, type in the following and press ENTER:

DISM.exe /Online /Cleanup-image /RestoreHealth

to repair the Windows image:

Deployment Image Servicing and Management tool
Version: 10.0.18362.1

Image Version: 10.0.18363.720

[==========================100.0%==========================] The restore operation completed successfully.
The operation completed successfully.

If RestoreHealth fails and you’re on Windows 10, then you should checkout this article:

https://www.tenforums.com/tutorials/16397-repair-install-windows-10-place-upgrade.html

on how to do an in place upgrade of Windows 10, opting to keep your settings and apps

5. If all goes well you’ll see that the restore operation completed successfully and you’ll be taken again to the command-line prompt C:\Windows\system32>

Now that the windows image is checked and fine, you should check your Windows installation against that image, giving the following command and pressing ENTER:

sfc /scannow

Beginning system scan.  This process will take some time.

Beginning verification phase of system scan.
Verification 100% complete.

Windows Resource Protection found corrupt files and successfully repaired them.
For online repairs, details are included in the CBS log file located at
windir\Logs\CBS\CBS.log. For example C:\Windows\Logs\CBS\CBS.log. For offline
repairs, details are included in the log file provided by the /OFFLOGFILE flag.

After any automatic repairs you should see the prompt C:\Windows\system32> again. Now repeat the same step till you see no more errors found and repaired.

sfc /scannow

Beginning system scan.  This process will take some time.

Beginning verification phase of system scan.
Verification 100% complete.

Windows Resource Protection did not find any integrity violations.

When back at the C:\Windows\system32> prompt with no errors found and repaired, just close the console window or type in the following and press ENTER:

exit

HowTo: Optimize your website enabling web server GZIP compression

Was recently checking out the fine tools from GTmetrix for optimizing some websites I manage and one of the issues I noticed on a client’s site was that GZIP compression was turned off.

They have a GZIP compression setup article for IIS and Apache web servers that contains a sample for Apache with compression rules for various mime types and exceptions for older Mozilla 4-era and MSIE browsers that may have issues with it.

There is also GZIP compression at NGINX

A simple tool to check a remote URL just for GZIP compression is at https://varvy.com/tools/gzip/ and at their read more page they list alternative ways to set up GZIP compression for Apache web servers (they do list fewer MIME type there, so do checkout the sample from GTmetrix too and combine) and an example for NGINX that also defines some extra rules for IE6 and exclusion for IE<6.

Fix: Microworlds Pro installer showing gibberish instead of Greek

A friend sent me the other day a screenshot from the installer of Microwords Pro Greek version on Windows 10 that was showing gibberish instead of Greek. At their school on Windows 10 it was showing up fine, so they wondered what the issue was.

They mentioned that on Windows 10 english it was showing fine. So we did try changing the preferred language order of Windows (at Laguage preferences in settings), to put English first (clicking on a language there shows arrows to reorder them, but has no drag action allowed btw), then Greek and try installing again, but the issue was still the same.

Note that at first I had wondered if it was a similar issue with the OwnCloud client installation that was showing an unexpected language instead of English, but since the screenshot was showing gibberish chars I guessed it was indeed showing Greek, but at some unexpected encoding for the system’s codepage. Note that codepages are a thing of the past and not needed for Unicode programs, but they’re still needed for older programs that just used a single byte (that couldn’t fit all posisble characters in the world) to encode a character.

I noticed it was showing a WinRar icon at that installer, so wondered if they had WinRar installed, but since installers are usually self-contained, I guessed the installer had been built with some older non-unicode WinRar engine version or something.

So I suggested going to Control Panel > Change Display Language > Administrative > Change System Locale for non Unicode programs and choose Greek there instead of English and indeed it worked.

Note that to find Control Panel on recent versions of Windows 10, the easiest way is to press the Search icon on the taskbar and then type “Control Panel” (or “Πίνακας Ελέγχου” if you’re on a Greek system). On the first Windows 10 editions you could just right click the Windows 10 Start/flag button on the taskbar and select Control Panel from the popup menu shown, but it seems they’ve removed that with recent updates.

This is useful to know since it usually affects most older non-Unicode programs that expect the system to work with a specific codepage. Microsoft used to have a utility that allowed one to switch the system codepage separately for various programs instead of doing a system-wide change, but it was hard to find even back then (plus didn’t allow redistribution) and most probably doesn’t work on newer systems.

Fix: Cisco Webex Meetings install fail (AddDllDirectory @ KERNEL32.dll)

Looking into the following error message occuring with Cisco WebEx Meetings installer on Windows 7, found this useful discussion:

https://social.technet.microsoft.com/Forums/en-US/a0970bfe-2bca-4ae3-a463-a5a04df83770/could-not-locate-dynamic-link-library-kernel32dll?forum=w7itproinstall

where the following are suggested:

– install Update for Windows 7 (KB2533623) from Microsoft:

https://www.microsoft.com/en-us/download/details.aspx?id=26767

– video tutorial:

https://www.youtube.com/watch?v=TpRRiMGJ_xA

---

And an extra tip, in case you after the installation, when you try to connect via a meeting URL, you see this dialog:

then try enabling TLS 1.1 & 1.2 at your browser (e.g. was told Chrome on Win7 had then both off). See how to do this for various browsers at: https://knowledge.digicert.com/generalinformation/INFO3299.html

Regarding TLS 1.1 though, mind you that it is considered insecure – so you might decide to skip enabling it (and try just enabling TLS 1.2) unless you can’t find some other solution. Quoting recent article on TLS 1.0 and 1.1 protocols:

Microsoft announced today that it will delay disabling support for the insecure Transport Layer Security (TLS) 1.0 and 1.1 protocols from Microsoft web browsers because of the current global situation until the second half of 2020, with an estimated time of roll out during July. “For the new Microsoft Edge (based on Chromium), TLS 1.0 and 1.1 are currently planned to be disabled by default no sooner than Microsoft Edge version 84 (currently planned for July 2020),” Kyle Pflug, Microsoft Edge Developer Experience Principal PM Lead, said. “For all supported versions of Internet Explorer 11 and Microsoft Edge Legacy (EdgeHTML-based), TLS 1.0 and TLS 1.1 will be disabled by default as of September 8, 2020.” https://www.bleepingcomputer.com/news/security/microsoft-delays-disabling-insecure-tls-in-browsers-until-july/

Fix: make ownCloud installer display in English language

OwnCloud is an interesting solution for setting up a file sharing cloud for a group of people.

However,one issue I’ve found with its Windows desktop client’s current version (which looks clean of any viruses since I always check first) is that if your Windows 10 is configured with a preferred language that the desktop client’s installer doesn’t have localization support for, then it doesn’t show up in English as you’d expect, but in Czech or someother language that most of us don’t know how to read.

So I tried running it’s MSI installer (ownCloud-2.6.1.13407.13049.msi) with –? parameter from the command-line and the /g languageCode parameter mentioned there looked promising, but trying /g en for English didn’t work. I guessed it needed some specific language code number (and not double-letter language code like en for English), since the help text was mentioning to see Windows Installer SDK for more help.

After a quick search I found an article that suggested passing the parameter Productlanguage=1033 to an msi installer on the command-line for it to ALWAYS show in English. And indeed it worked.

To open a command window one can click the Search icon on the windows taskbar and type CMD then press ENTER.

Then they can drag-drop the .MSI file of ownCloud installer onto the black command-line window that opens up and type an extra space char and then Productlanguage=1033 before pressing ENTER to launch the ownCloud installer in English. After that they can close the command-line window at anytime.

Since many users may be uncomfortable with such instructions, one could provide an msiEnglish.bat file that just contains

%1 Productlanguage=1033

User could drag-drop the .msi they want onto that msiEnglish.bat file and it would run the msi installer being displayed in English language, irrespective of any preferred language settings at the Windows operating system.

Of course the best thing would be if ownCloud fixed their desktop client installer to fallback to the Engish language (set it as default) if it can’t find localization strings for the currently prefered language of the user. Have filed an issue at https://github.com/owncloud/client/issues/7825

Fix: Buildbox activation issues on load

Buildbox is a wonderful game authoring tool (that spans the whole nocode-code continuum, including the low-code aspect). It now has a free version too with nice templates and tutorials included.

I had an issue with the Windows version (it also has an indentical MacOS-X version since it’s a Qt-based app) where it was complaining about activation issue at startup and couldn’t proceed.

To solve that you can remove activation info (which won’t be removed if you just do uinstall and reinstall) by deleteing the com.eightcell.buildbox subfolder at:

C:\Users\YOUR_PROFILE_NAME_HERE\AppData\Local\eightcell\Buildbox\

Next time you run Buildbox.exe it will show activation dialog and you can enter your registration key (even the free version has one that you can download from your free account at http://buildbox.com)

With the latest Buildbox 3 there seems to exist an extra com.eightcell.buildbox3 subfolder that you could delete or just rename the main.iblicense files in there to main._iblicense. The launch Buidlbox again to see the activation dialog.

There probably exists some bug in Buildbox that causes one to repeat this action often.
So you could add a RESET_BUILDBOX_LICENSE.BAT file to your desktop with the following commands to delete the license file:

del "%AppData%\..\Local\eightcell\Buildbox\com.eightcell.buildbox3\main.iblicense"
@pause