Archive

Posts Tagged ‘HowTo’

HowTo: Extract numeric suffix from a string in Python

I recently needed to extract a numeric suffix from a string value in Python. Initially I did the following:

import re

def extractNumSuffix(value):

    return (None if (search:=re.search(‘^\D*(\d+)$’, value, re.IGNORECASE)) is None else search.group(1))

Note that return has a single-line expression.

So

print(extractNumSuffix("test1030"))

prints 1030

Tried it online at:

https://repl.it/languages/python3

extractNumSuffix

However, then I found out that Assignment Expressions in Python only work from Python 3.8 and up, so I changed it to this one:

import re

def extractNumSuffix(value):
    search=re.search(r"^\D*(\d+)$", value, re.IGNORECASE)
    return (None if search is None else search.group(1))

which should work in Python 2.x too. Don’t forget to import the regular expressions (re) module.

HowTo: Install Python, PIP and cx_Oracle on MSYS2 / Windows

Here’s how to install Python, PIP package installer and cx_Oracle extension module on MSYS2:

$ pacman -S python
warning: python-3.8.3-1 is up to date — reinstalling
resolving dependencies…
looking for conflicting packages…

Packages (1) python-3.8.3-1

Total Installed Size:  108.66 MiB
Net Upgrade Size:        0.00 MiB

:: Proceed with installation? [Y/n] y

$ pacman -S cx_Oracle
error: target not found: cx_Oracle

$ pip install cx_Oracle
bash: pip: command not found

$ pacman -S python-pip
resolving dependencies…
looking for conflicting packages…

Packages (2) python-setuptools-47.1.1-1  python-pip-20.1.1-1

Total Download Size:    2.22 MiB
Total Installed Size:  10.91 MiB

:: Proceed with installation? [Y/n] y
:: Retrieving packages…

$ pip install cx_Oracle
Collecting cx_Oracle
  Downloading cx_Oracle-8.0.0.tar.gz (325 kB)
     |████████████████████████████████| 325 kB 693 kB/s
Using legacy setup.py install for cx-Oracle, since package ‘wheel’ is not installed.
Installing collected packages: cx-Oracle
    Running setup.py install for cx-Oracle … error
    ERROR: Command errored out with exit status 1:
     command: /usr/bin/python3.exe -u -c ‘import sys, setuptools, tokenize; sys.argv[0] = ‘"’"’/tmp/pip-install-wqtzv2rv/cx-Oracle/setup.py’"’"’; __file__=’"’"’/tmp/pip-install-wqtzv2rv/cx-Oracle/setup.py’"’"’;f=getattr(tokenize, ‘"’"’open’"’"’, open)(__file__);code=f.read().replace(‘"’"’\r\n’"’"’, ‘"’"’\n’"’"’);f.close();exec(compile(code, __file__, ‘"’"’exec’"’"’))’ install –record /tmp/pip-record-mui2mjs9/install-record.txt –single-version-externally-managed –compile –install-headers /usr/include/python3.8/cx-Oracle
         cwd: /tmp/pip-install-wqtzv2rv/cx-Oracle/
    Complete output (17 lines):
    running install
    running build
    running build_ext
    building ‘cx_Oracle’ extension
    creating build
    creating build/temp.msys-3.1.5-x86_64-3.8
    creating build/temp.msys-3.1.5-x86_64-3.8/src
    creating build/temp.msys-3.1.5-x86_64-3.8/odpi
    creating build/temp.msys-3.1.5-x86_64-3.8/odpi/src
    x86_64-pc-msys-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O2 -pipe -march=x86-64 -mtune=generic -O2 -pipe -march=x86-64 -mtune=generic -O2 -pipe -DCXO_BUILD_VERSION=8.0.0 -Iodpi/include -Iodpi/src -I/usr/include/python3.8 -c src/cxoApiType.c -o build/temp.msys-3.1.5-x86_64-3.8/src/cxoApiType.o
    In file included from src/cxoModule.h:14,
                     from src/cxoApiType.c:11:
    /usr/include/python3.8/Python.h:44:10: fatal error: crypt.h: No such file or directory
       44 | #include <crypt.h>
          |          ^~~~~~~~~
    compilation terminated.
    error: command ‘x86_64-pc-msys-gcc’ failed with exit status 1
    —————————————-
ERROR: Command errored out with exit status 1: /usr/bin/python3.exe -u -c ‘import sys, setuptools, tokenize; sys.argv[0] = ‘"’"’/tmp/pip-install-wqtzv2rv/cx-Oracle/setup.py’"’"’; __file__=’"’"’/tmp/pip-install-wqtzv2rv/cx-Oracle/setup.py’"’"’;f=getattr(tokenize, ‘"’"’open’"’"’, open)(__file__);code=f.read().replace(‘"’"’\r\n’"’"’, ‘"’"’\n’"’"’);f.close();exec(compile(code, __file__, ‘"’"’exec’"’"’))’ install –record /tmp/pip-record-mui2mjs9/install-record.txt –single-version-externally-managed –compile –install-headers /usr/include/python3.8/cx-Oracle Check the logs for full command output.

Found a solution here for other Linuxes, mentioning

apt install python-dev clang libcrypt-dev

Via MSYS2 package manager (pacman), libcrypt and libcrypt-devel seem to be available. The second one should fetch the first one if needed, together with header files for development.

$ pacman -S libcrypt-devel
resolving dependencies…
looking for conflicting packages…

Packages (1) libcrypt-devel-2.1-2

Total Download Size:   0.04 MiB
Total Installed Size:  0.04 MiB

:: Proceed with installation? [Y/n] y

Now we can install cx_Oracle succesfully:

$ pip install cx_Oracle
Collecting cx_Oracle
  Using cached cx_Oracle-8.0.0.tar.gz (325 kB)
Using legacy setup.py install for cx-Oracle, since package ‘wheel’ is not installed.
Installing collected packages: cx-Oracle
    Running setup.py install for cx-Oracle … done
Successfully installed cx-Oracle-8.0.0

HowTo: make raw_input & input work the same in both Python 2 and 3

Was just trying to make a Python 2 script work in Python 3 and at first it seemed I just needed a small change to add missing parentheses to the argument of a couple of print statements.

But then another issue came up, it wasn’t understanding the command raw_input(somePrompt) that was occuring at various places in that file to input text from the console.

Various solutions were proposed at

https://stackoverflow.com/questions/954834/how-do-i-use-raw-input-in-python-3/62825723

but I think I came up with a cleaner looking solution that works in both Python2 and Python3, that should allow one to use either raw_input or input in both Python2 and Python3 with the semantics of Python2’s raw_input (aka the semantics of Python3’s input).

# raw_input isn't defined in Python3.x, whereas input wasn't behaving 
# like raw_input in Python 2.x. This should make both input and raw_input 
# work in Python 2.x/3.x like the raw_input from Python 2.x 
try: input = raw_input
except NameError: raw_input = input

In practice this came up from ideas at other answers on that SO thread. It tries to define input as raw_input which should fail in Python3.x since raw_input is undefined. In that case it will catch a NameError exception and do the reverse, aka define raw_input as input. However in Python2 the first command should execute fine, overriding the insecure input of Python2.x to work the same as raw_input (not trying to interpret input strings that is). Actually that is what Python 3.x input does.

Wonder why they didn’t declare it like that in the first place though, breaking compilation of Python2 programs. After all semantically only Python2 programs that were using the old insecure input would have an issue, not those that were using raw_input which is the semantics Python3 promotes with its newer input.

HowTo: Use DISM and SFC tools to check+fix your Windows installation

If you’re having issues with your Windows 7 or newer, you should consider whether its installation has become corrupted (due to malicious software or hard drive errors).

After doing a disk check (say by right clicking the appropriate drive under my computer and selecting Properties, then Tools tab and Error checking) and a complete virus scan (on Win10 you can click the shield icon of Windows defender in the taskbar tray and at scanning options choose to do a full scan – or if you have installed some third-party antivirus double-click its icon in the taskbar tray and when its GUI opens up opt to do a full scan), then try the following steps to repair your Windows installation:

1. Press WIN+R to open Run dialog

2. Type in:

CMD

Hold down CTRL+SHIFT keys and click OK to open the command line window in Administrator mode (do press Yes at the User Account Control prompt)

A (usually) black text-based console window will open up and you’ll be greated with something like:

Microsoft Windows [Version 10.0.18363.720]
(c) 2019 Microsoft Corporation. All rights reserved.

and then a prompt like:

C:\Windows\system32>

3. Type in the following and press the ENTER key:

DISM.exe /Online /Cleanup-image /Scanhealth

and press ENTER to execute the DISM tool with the option to check the windows image health and wait patiently for it to complete

Deployment Image Servicing and Management tool
Version: 10.0.18362.1

Image Version: 10.0.18363.720

[==========================100.0%==========================] The component store is repairable.
The operation completed successfully.

4. In case you see a message that the component store is repairable, then when greeted with the C:\Windows\system32> prompt again, type in the following and press ENTER:

DISM.exe /Online /Cleanup-image /RestoreHealth

to repair the Windows image:

Deployment Image Servicing and Management tool
Version: 10.0.18362.1

Image Version: 10.0.18363.720

[==========================100.0%==========================] The restore operation completed successfully.
The operation completed successfully.

If RestoreHealth fails and you’re on Windows 10, then you should checkout this article:

https://www.tenforums.com/tutorials/16397-repair-install-windows-10-place-upgrade.html

on how to do an in place upgrade of Windows 10, opting to keep your settings and apps

5. If all goes well you’ll see that the restore operation completed successfully and you’ll be taken again to the command-line prompt C:\Windows\system32>

Now that the windows image is checked and fine, you should check your Windows installation against that image, giving the following command and pressing ENTER:

sfc /scannow

Beginning system scan.  This process will take some time.

Beginning verification phase of system scan.
Verification 100% complete.

Windows Resource Protection found corrupt files and successfully repaired them.
For online repairs, details are included in the CBS log file located at
windir\Logs\CBS\CBS.log. For example C:\Windows\Logs\CBS\CBS.log. For offline
repairs, details are included in the log file provided by the /OFFLOGFILE flag.

After any automatic repairs you should see the prompt C:\Windows\system32> again. Now repeat the same step till you see no more errors found and repaired.

sfc /scannow

Beginning system scan.  This process will take some time.

Beginning verification phase of system scan.
Verification 100% complete.

Windows Resource Protection did not find any integrity violations.

When back at the C:\Windows\system32> prompt with no errors found and repaired, just close the console window or type in the following and press ENTER:

exit

HowTo: Optimize your website enabling web server GZIP compression

Was recently checking out the fine tools from GTmetrix for optimizing some websites I manage and one of the issues I noticed on a client’s site was that GZIP compression was turned off.

They have a GZIP compression setup article for IIS and Apache web servers that contains a sample for Apache with compression rules for various mime types and exceptions for older Mozilla 4-era and MSIE browsers that may have issues with it.

There is also GZIP compression at NGINX

A simple tool to check a remote URL just for GZIP compression is at https://varvy.com/tools/gzip/ and at their read more page they list alternative ways to set up GZIP compression for Apache web servers (they do list fewer MIME type there, so do checkout the sample from GTmetrix too and combine) and an example for NGINX that also defines some extra rules for IE6 and exclusion for IE<6.

HowTo: disable video autoplay in Chromium-based Microsoft Edge

Getting really annoyed by YouTube’s insistence to autoplay (and not even stop the previously playing video) everytime you navigate to a browser tab that shows YouTube content (say using CTRL+TAB / SHIFT+CTRL+TAB to find a tab you’re looking for when you have too many and they only manage to fit their icons so no title to pick one)?

The way to stop it in the newest Microsoft Edge browser (that’s based on the Chromium engine, same one that Google Chrome uses), is to press the three dots button at the top-right and select Settings.

image

When at Settings, select “Site permissions”, scroll down and click the arrow button on the far right to open “Media autoplay”

image

Finally, select “Limit” from the dropdown menu.

image

And you’re done.

HowTo: Reset browser cache of CSS files upon ASP.net MVC app publish

On an ASP.net MVC webapp I’m maintaining, I had the issue that due to caching of older CSS (stylesheet) files in the browser, if the user didn’t press F5/refresh, it wasn’t showing you some message (since I had added the class .center-horiz-vert in the CSS that didn’t exist in the older cached css the browser had).

Instead of changing web.config to stop cachine of CSS files (in which case it would bring the CSS on every page load which is an overkill), I expanded on an idea mentioned by Maxim Kornilov on SO (https://stackoverflow.com/a/12992813/903783), on making the CSS URLs webapp version specific.

I added a fake version parameter to the URLs with the build number as value so that till I publish a new build the browser caches the CSS, but when I upload a new build it brings the new one since it cache with the url as a key (that now includes the build number as a dummy url parameter that the webserver will ignore and just fetch the CSS file when requested)

Maxim’s example was in ASP/ASP.net WebForms syntax instead of MVC’s and Razor Pages’ newer Razor syntax), so I contributed my solution for the case of an ASP.net MVC webapp that wants to serve a fresh copy of CSS files on every new build that you publish (will do this whether the CSS file has changed or not) so that browsers don’t use older cached copies of the file. Obviously this expands to any kind of files you link/load into your webpages via a URL.

1) Added to the webapp’s main class (was called MvcApplication) in Global.asax.cs

#region Versioning

public static string Version => 
typeof(MvcApplication).Assembly.GetName().Version.ToString();
//note: syntax requires C# version >=6 public static DateTime LastUpdated =>
File.GetLastWriteTime(typeof(MvcApplication).Assembly.Location); #endregion

the someProperty => someReadOnlyExpression syntax is just shorthand for someProperty { get { return … ;} } possible since C# 6

2) in its Content/_Layout.cshtml file I used to have the following to show build number and build datetime (based on the webapp’s main assembly) on the page footer:

Version @ViewContext.Controller.GetType().Assembly.GetName().Version 
(@string.Format("{0:yyyy/MM/dd-HH:mm:ss}",
@File.GetLastWriteTime(ViewContext.Controller.GetType().Assembly.Location)))

which I changed to the simpler:

Version @somewebappname.MvcApplication.Version
(@string.Format("{0:yyyy/MM/dd-HH:mm:ss}",
somewebappname.MvcApplication.LastUpdated))

3) it was loading the CSS via hardcoded link in _Layout.cshtml (still refactoring it) which I changed to:

<link href='@Url.Content("~/Content/Site.css?version=" + 
somewebappname.MvcApplication.Version)' rel="stylesheet" type="text/css" />

so if one right-clicks in the webpage and they do view source they see:

<link href='/Content/Site.css?version=2.1.5435.22633' 
rel="stylesheet" type="text/css" />

that is the CSS url is version specific thanks to the dummy parameter version

If a random number was used instead it would fetch the CSS at every page load which is usually undesired, especially if you are already pushing a new webapp build instead of individual page changes to the web server (so that you do have access to a build number that you can inject into URLs).

Note that to achieve auto-incrementing of build number, at Properties/AssemblyInfo.cs I have (see How to have an auto incrementing version number (Visual Studio)?):

// Version information for an assembly consists of the following four values:
//
//      Major Version
//      Minor Version 
//      Build Number
//      Revision
//
// You can specify all the values or you can default the Revision 
// and Build Numbers by using the '*' as shown below: [assembly: AssemblyVersion("1.0.*")] //[assembly: AssemblyFileVersion("1.0.*")]
// don't use boh AssemblyVersion and AssemblyFileVersion with auto-increment

Fix: Invalid Firmware Image at Dell Inspiron 3537 BIOS update

For some time now I was trying to update the BIOS of an older Dell Inspiron 3537 laptop from inside Windows (with the InsydeFlash application that the respective Dell update package employs), only to get a blue screen saying Invalid Firmware Image upon reboot and the BIOS update was skipped every time.

Since that update was fixing a critical security issue (Intel Security Advisory INTEL-SA-00115 – CVE-2018-3639 & CVE-2018-3640), I decided to do some more research on it. I eventually came to the conclusion that since I had A9 BIOS version, I needed to install BIOS version A10 first (which addresses CVE-2017-5715 and associated Intel Reboot issue), then try the BIOS version A11 update that the Dell Support online was offering.

Luckily there was a Dell BIOS update guide that was suggesting to visit Dell downloads catalog to find older updates, from where I found all Dell Inspiron 3537 updates and was able to locate the A10 BIOS update.

Updating the BIOS from version A9 (the version I had, as displayed at “Current BIOS” field on the InsydeFlash UI) to A10 and then after reboot from A10 to A11 with the respective update executables worked fine. Can confirm the update is done by launching the update once more and then just Cancel.

image

HowTo: round a number up to N decimal digits in Javascript

Was just trying to round-off some Google Maps coordinates for display in Javascript up to 3 decimal digits and that was a bit like a blast from the past (the end of the ‘90s to be more accurate)…

So here’s my contributed answer at:
https://stackoverflow.com/questions/2221167/javascript-formatting-a-rounded-number-to-n-decimals

This works for rounding to N digits (if you just want to truncate to N digits remove the Math.round call and use the Math.trunc one):

function roundN(value, digits) {
   var tenToN = 10 ** digits;
   return /*Math.trunc*/(Math.round(value * tenToN)) / tenToN;
}

Had to resort to such logic at Java in the past when I was authoring data manipulation E-Slate components. That is since I had found out that adding 0.1 many times to 0 you’d end up with some unexpectedly long decimal part (this is due to floating point arithmetics).

A user comment at Format number to always show 2 decimal places calls this technique scaling.

Some mention there are cases that don’t round as expected and at http://www.jacklmoore.com/notes/rounding-in-javascript/ this is suggested instead:

function round(value, decimals) {
  return Number(Math.round(value+'e'+decimals)+'e-'+decimals);
}

image

HowTo: Use latest C# features in MVC5 Razor views (.cshtml)

Having recently updated an ASP.net MVC web app from MVC4 to MVC5 and from .NET 4.5 to .NET 4.7.2 I was expecting Razor views (.cshtml files) to use the latest C# compiler, especially since at Properties/Build/Advanced option for the web project one read “C# latest major version (default)”.

However that was not the case and trying to use newer C# language features like the ?. ternary conditional operator or interpolated strings (or nameof etc.) would show errors like

Feature ‘interpolated strings’ is not available in C# 5. Please use language version 6 or greater.

Luckily there is a workaround for using the latest C# compiler in MVC5. Just need to add the NuGet package https://www.nuget.org/packages/Microsoft.CodeDom.Providers.DotNetCompilerPlatform/ to one’s project as explained at https://dusted.codes/using-csharp-6-features-in-aspdotnet-mvc-5-razor-views. Alternatively one could move their project to ASP.net Core, which is a more drastic move though.

After doing it I started seeing Intellisense issues in .cshtml like:

The type ‘Expression<>’ is defined in an assembly that is not referenced. You must add a reference to assembly ‘System.Core …

Tried to add the System.Core assembly to the project, but wasn’t allowed (it said the Build system was adding it). Adding System.Core as a NuGet package would mean moving to .NET Core which I wasn’t ready to try with that project yet.

Seems there was an easy solution to that, just closed and reopened the Visual Studio solution and did a Rebuild and all was fine after that.

Categories: Posts Tags: , , , , , ,
%d bloggers like this: