When you run Magento 2 behind Varnish on a busy store, cache invalidation can easily become a performance problem.

Traditionally, when a Varnish object (page) gets purged, the next user who hits that page pays the price: Varnish has to fetch from the backend, generate a new response, and only then cache it.

On low-traffic sites this isn’t a huge issue, but on high-traffic stores it becomes painful very quickly:

• The first user after a purge gets a slow response
• If 10 users hit the same page while the cache is rebuilding, you get 10 slow responses
• If you purge often (e.g. frequent product updates), the site feels slow even though Varnish is technically enabled

The solution: Soft Purge using vmod_xkey.

Instead of immediately deleting objects from the cache, Varnish can:

• mark objects as stale,
• continue serving them to users,
• refresh a new version from the backend in the background.

Result: users still see a cached page, while Varnish quietly builds the fresh version.

In this guide we’ll configure Soft Purge for Magento 2 on Ubuntu with Varnish + xkey.

We’ll cover:

1. Installing varnish-modules (xkey) on Ubuntu
2. Enabling import xkey; in your VCL
3. Replacing the default Magento BAN logic with xkey soft purge
4. Setting grace so Varnish can safely serve stale content
5. Testing soft purge with curl

Examples assume Magento 2 + Varnish on Ubuntu.

1. Install varnish-modules (xkey) on Ubuntu

On Ubuntu, the vmod_xkey module is provided by the varnish-modules package.

sudo apt update  
sudo apt install varnish-modules  

Check that the xkey vmod is available (path may vary by distro and Varnish version):

ls -1 /usr/lib/varnish/vmods  
# You should see something like: libvmod_xkey.so

Note: if you’re using a custom Docker image or a containerized Varnish setup, vmod_xkey must be built into that image. In some managed environments it’s already included by default.

2. Enable xkey in your VCL

In your main VCL file (often /etc/varnish/default.vcl, or an entry-point VCL that includes the Magento-generated file), add this at the top:

import xkey;  

Example:

vcl 4.1;

import std;  
import xkey;

backend default {  
    .host = "127.0.0.1";
    .port = "8080";
}

If you’re using the Magento-generated VCL (magento2.vcl), make sure import xkey; is added to the actual VCL file loaded by Varnish, not just some unused include.

3. Replace Magento BAN logic with xkey Soft Purge

Magento 2 invalidates cache via HTTP BAN (or PURGE) requests that carry the header X-Magento-Tags-Pattern. In a typical VCL you’ll see something like:

if (req.method == "BAN") {  
    if (req.http.X-Magento-Tags-Pattern) {
        ban("obj.http.X-Magento-Tags ~ " + req.http.X-Magento-Tags-Pattern);
    }
    return (synth(200, "Banned"));
}

With soft purge we keep the idea of tag-based invalidation, but instead of hard-removing objects, we mark them as stale using xkey.softpurge().

In sub vcl_recv, in the PURGE/BAN section, update it to something like this (adapt as needed to match your existing structure):

sub vcl_recv {  
    # ... your existing logic (health checks, static assets, etc.)

    if (req.method == "PURGE") {
        # Full Page Cache flush – still a hard ban
        if (req.http.X-Magento-Tags-Pattern == ".*") {
            ban("obj.http.X-Magento-Tags ~ " + req.http.X-Magento-Tags-Pattern);
        }
        # Soft purge for specific tags
        elseif (req.http.X-Magento-Tags-Pattern) {
            # Example: "((^|,)cat_c(,|$))|((^|,)cat_p(,|$))" → "cat_c cat_p"
            set req.http.X-Magento-Tags-Pattern =
                regsuball(req.http.X-Magento-Tags-Pattern, "[^a-zA-Z0-9_-]+", " ");

            # Trim spaces
            set req.http.X-Magento-Tags-Pattern =
                regsuball(req.http.X-Magento-Tags-Pattern, "(^\\s*)|(\\s*$)", "");

            # Soft purge via xkey
            set req.http.n-gone = xkey.softpurge(req.http.X-Magento-Tags-Pattern);

            return (synth(200, "Invalidated " + req.http.n-gone + " objects"));
        }

        return (synth(200, "Purged"));
    }

    # ... rest of vcl_recv
}

What this does:

• Takes the original X-Magento-Tags-Pattern (which is usually a regex) and turns it into a space-separated list of tags (cat_c cat_p, cat_p_123, etc.). That’s the format xkey expects.
• Calls xkey.softpurge() to mark all matching objects as stale instead of fully purged.
• Returns a synthetic 200 response like Invalidated 5 objects – very handy for debugging with curl.

For a full flush (X-Magento-Tags-Pattern: .*) we still use a classic ban(). In most Magento use cases, that’s exactly what you want when doing a global FPC flush.

4. Configure grace and xkey tags in vcl_backend_response

Soft purge is only useful if Varnish is allowed to serve stale content while it fetches a fresh version in the background. We control that with beresp.grace.

In sub vcl_backend_response, add or update something like this:

sub vcl_backend_response {  
    # Time window during which stale content can be served
    set beresp.grace = 3h;

    # Use xkey based on Magento tags
    if (beresp.http.X-Magento-Tags) {
        # Expose grace for debugging
        set beresp.http.Grace = beresp.grace;

        # Turn comma-separated X-Magento-Tags into a space-separated list for xkey
        set beresp.http.xkey = regsuball(beresp.http.X-Magento-Tags, ",", " ");

        # Optionally reset X-Magento-Tags to a generic value
        # so you don't leak massive tag lists further downstream
        set beresp.http.X-Magento-Tags = "fpc";
    }

    # ... rest of vcl_backend_response
}

Key points:

• beresp.grace = 3h; tells Varnish: even after TTL expires, you may serve this object as stale for up to 3 hours while you fetch a fresh version.
• beresp.http.xkey is populated from Magento’s X-Magento-Tags header. That’s what xkey.softpurge() uses to find and mark the right objects as stale.
• Be careful not to duplicate beresp.grace if your Magento-generated VCL already sets it. Adjust the existing line instead of adding a second one.

5. Reload Varnish with the new VCL

After editing your VCL:

sudo systemctl reload varnish  
# or, if reload is not configured:
sudo systemctl restart varnish  

Confirm that the active VCL is loaded:

sudo varnishadm vcl.list  

6. Testing Soft Purge

6.1. Check tags and xkey headers

First, make a normal GET request to a Magento page (for example, a product page) and inspect the headers:

curl -I https://example.com/some-product.html  

You should see something along the lines of:

• X-Magento-Tags: cat_p_169745,cat_c_23,...
• xkey: cat_p_169745 cat_c_23 ...
• Grace: 3h

Header names can vary slightly depending on your VCL, but the important part is that xkey is populated with Magento’s tag list and Grace is set.

6.2. Soft purging a specific tag

Now trigger a soft purge via PURGE + X-Magento-Tags-Pattern:

curl -k -X PURGE \  
  -H 'X-Magento-Tags-Pattern: cat_p_169745' \
  http://varnish.internal:6081/some-product-url

If everything is wired correctly, you’ll get a synthetic response like:

<!DOCTYPE html>  
<html>  
<head>  
<title>200 Invalidated 1 objects</title>  
</head>  
<body>  
<h1>Error 200 Invalidated 1 objects</h1>  
<p>Invalidated 1 objects</p>  
<h3>Guru Meditation:</h3>  
<p>XID: 65566</p>  
<hr>  
<p>Varnish cache server</p>  
</body>  
</html>  

Important details:

• HTTP status is 200 (this is not a real error, just Varnish’s synthetic page style).
• The text says Invalidated 1 objects → xkey.softpurge() has marked 1 object as stale.

6.3. Observing behavior before/after soft purge

Typical flow:

1. Request before soft purge → HIT, fast response.
2. After soft purge → first client still gets a fast response (HIT with stale content), while Varnish refreshes the backend in the background.
3. Subsequent requests → now get the fresh version from the updated backend, still as cache HIT.

If you see MISS and slow responses instead, double-check:

• Is beresp.grace actually set (and not overridden later)?
• Does some other part of your VCL do return (pass); for that URL?
• Is some custom Magento/Varnish module doing extra purges or cache-bypass logic?

7. Notes for Production Environments

A few practical tips before rolling this into production:

• If you’re using a Magento Varnish extension (e.g. Jetrails Varnish extension or similar):

  • Always upgrade it to the latest version first
  • Regenerate the VCL from Magento
  • Then apply the soft purge modifications on top of that generated VCL

• If you run a multi-node Varnish cluster:

  • Make sure purge/softpurge is executed on all nodes (via shared admin endpoint, orchestration, or varnishadm over SSH).

• Be reasonable with grace values:

  • 3h is often a sweet spot for most e‑commerce sites
  • For extremely dynamic content, you might want shorter grace (e.g. 30–60 minutes)
  • You can also tune grace by path (e.g. longer grace for category pages, shorter for carts/checkout which should normally bypass Varnish anyway)

8. References

• Varnish "Grace" documentation:
  https://varnish-cache.org/docs/trunk/users-guide/vcl-grace.html

• vmod_xkey on GitHub:
  https://github.com/varnish/varnish-modules/blob/master/src/vmod_xkey.vcc

If you’re debugging a high-traffic Magento 2 store and notice that Varnish is enabled but your users still see slow page loads after product updates, switching from hard purges to soft purges with xkey is one of the highest ROI changes you can make.

So today I spent some quality time debugging a frustrating ReCaptcha issue on a Magento 2 store. If you've ever seen that dreaded "Invalid argument" message inside the ReCaptcha widget, this post is for you.

The Problem

A customer contacted me about an issue on their production Magento 2.4.8 store - the login page was showing a strange error in the ReCaptcha widget. Instead of the normal "protected by reCAPTCHA" badge, there was a red "Invalid argument" message.

To debug this safely without touching production, I cloned the entire store (database and files) to our staging environment using Magento 2 Gitpod on ona.com (https://github.com/nemke82/magento2gitpod). The issue reproduced perfectly on staging - same error, same behavior.

I created fresh Google ReCaptcha v3 keys specifically for the staging domain and configured them in Magento admin. But the error persisted.
The weird part? I tested the same ReCaptcha keys on a fresh Magento 2.4.8 installation and they worked perfectly. So the issue was specific to this codebase, not the keys or domain configuration.

The ReCaptcha widget was there, but instead of the nice "protected by reCAPTCHA" badge, I got this ugly red error message.
The weird part? I tested the same ReCaptcha keys on a fresh Magento 2.4.8 installation and they worked perfectly. So the issue was specific to this codebase.

The Investigation

First stop - Magento logs. I checked var/log/exception.log and var/log/system.log. Found some interesting errors about a module called Mageplaza_GoogleRecaptcha:

Class "Mageplaza\GoogleRecaptcha\Model\System\Config\Source\Frontend\Forms" not found  
Class "Mageplaza\GoogleRecaptcha\Observer\Captcha" does not exist  

Hmm. The module was being called but didn't exist. 🤔
I assume customer had module, removed it somehow but always that leftovers... meh Magento...

Root Cause #1: Orphaned Module Configuration

Turns out, the Mageplaza GoogleRecaptcha extension was previously installed on this store but later removed. The problem? The configuration data was still sitting in the database:

SELECT * FROM core_config_data WHERE path LIKE 'googlerecaptcha%';  

config_id  scope     scope_id  path                                      value                                         updated_at  
1379       default   0         googlerecaptcha/general/enabled           1                                             2026-01-16 12:24:41  
1380       default   0         googlerecaptcha/general/language          sr                                            2026-01-16 12:24:41  
1381       default   0         googlerecaptcha/general/invisible/api_key NULL                                          2026-01-16 12:27:21  
1382       default   0         googlerecaptcha/general/invisible/api_secret NULL                                       2026-01-16 12:27:21  
1383       default   0         googlerecaptcha/general/visible/api_key   0:3:SijGQlxN...encrypted...                   2026-01-16 12:27:21  
1384       default   0         googlerecaptcha/general/visible/api_secret 0:3:SNydl+LZ...encrypted...                  2026-01-16 12:27:21  
1385       default   0         googlerecaptcha/backend/enabled           0                                             2024-10-31 16:24:06  
1386       default   0         googlerecaptcha/frontend/enabled          1                                             2026-01-16 12:24:41  
1387       default   0         googlerecaptcha/frontend/type             visible                                       2026-01-16 12:24:41  
1388       default   0         googlerecaptcha/frontend/forms            body.customer-account-login #login-form...    2026-01-16 12:24:41  
1389       default   0         googlerecaptcha/frontend/position         0                                             2026-01-14 21:23:20  
1390       default   0         googlerecaptcha/frontend/theme            dark                                          2026-01-16 12:24:41  
2048       default   0         googlerecaptcha/frontend/size             compact                                       2026-01-16 12:24:41  

The config showed googlerecaptcha/frontend/enabled = 1 - so Magento was trying to use a module that no longer existed!

Root Cause #2: Invalid Language Code

While digging through the config, I noticed something else odd. The ReCaptcha language setting was set to "0":

recaptcha_frontend/type_recaptcha_v3/lang = 0  

** Now, for english default this is fine, but this customer switched frontend to custom Language (Serbian)

That's not a valid language code! It should be something like "en", "sr", or empty string for auto-detect. This invalid value was being passed to Google's API as hl=0, causing the "Invalid argument" error.

path                                        value  
recaptcha_frontend/type_recaptcha/lang      0  
recaptcha_frontend/type_invisible/lang      0  
recaptcha_frontend/type_recaptcha_v3/lang   0  
recaptcha_backend/type_recaptcha/lang       0  
recaptcha_backend/type_recaptcha_v3/lang    0  

The Fix

Two simple database updates fixed everything:
1. Disable the orphaned Mageplaza config:

UPDATE core_config_data SET value = '0' WHERE path = 'googlerecaptcha/frontend/enabled';  
UPDATE core_config_data SET value = '0' WHERE path = 'googlerecaptcha/general/enabled';  

1. Fix the invalid language codes:

UPDATE core_config_data SET value = '' WHERE path LIKE '%recaptcha%lang%' AND value = '0';  

1. Clear the cache (or roll over deployment entirely):

php bin/magento cache:flush  

And just like that...

The beautiful blue "protected by reCAPTCHA" badge appeared! 🎉

Lessons Learned

Always clean up after removing extensions - Don't just delete the module files. Remove the config data from coreconfigdata table too.
Check for module conflicts - When you have multiple ReCaptcha solutions (Magento native + third-party), they can conflict even if one is "disabled".
Validate your config values - A simple "0" instead of empty string or proper value can break things in unexpected ways.
Fresh installs are your friend - Testing on a clean Magento installation helped confirm the issue was codebase-specific, not a key/domain problem.

Quick Debug Checklist for ReCaptcha Issues
If you're seeing "Invalid argument" in your Magento ReCaptcha:

[ ] Check coreconfigdata for orphaned third-party ReCaptcha configs
[ ] Verify language codes aren't set to invalid values like "0"
[ ] Ensure only ONE ReCaptcha solution is active
[ ] Verify your keys match the ReCaptcha type (v2 vs v3)
[ ] Confirm your domain is registered in Google ReCaptcha console

Hope this saves someone a few hours of debugging!

TL;DR: I spent a day building VeloServe, a high-performance web server with embedded PHP support, using Cursor Pro and Claude Opus 4.5. It's now open source and you can try it in one command.

Visit: https://www.veloserve.io for more details.

The Beginning of a Wild Idea

I've been frustrated with the traditional web server stack for years. Nginx is great, Apache is reliable, but setting up PHP-FPM, configuring sockets, tuning workers... it's a lot. What if there was a single binary that just worked and at same time you could use it to scale with any Cloud provider or data center or at local dev purpose.

That's when I decided to build VeloServe — a modern web server written in Rust with PHP embedded directly into it. No PHP-FPM. No separate processes. Just speed.

The Setup: Cursor Pro + Claude Opus 4.5 + Ona.com

Here's my development setup that made this possible:

Cursor Pro with Remote SSH

I use Cursor as my primary IDE. It's VS Code on steroids with AI built-in. The game-changer? Remote SSH connections. I connected Cursor to an Ona.com workspace (formerly Gitpod), which gave me a fully configured development environment in the cloud.

Why Ona.com? It spins up a complete Linux environment with Docker, all the tools I need, and most importantly — I can close my laptop and pick up exactly where I left off from any device.

Claude Opus 4.5 as My Pair Programmer

The real magic happened with Claude Opus 4.5 through Cursor's AI features. But here's the thing — I didn't just blindly accept AI suggestions.

Every piece of Rust code, I verified against the official Rust documentation. Every Tokio async pattern, I cross-referenced with the Tokio docs. Every Hyper HTTP handling, I checked against Hyper's examples.

This is what I call Vibe Coding — you work with the AI, not for it. The AI suggests, you verify, you refine, you ship.

VeloServe is a web server that:

• Runs PHP inside itself — no external PHP-FPM process
• Written in Rust — memory safe, blazing fast
• Supports two modes:
  • CGI Mode — uses php-cgi, works everywhere
  • SAPI Mode — PHP embedded via FFI, 10-100x faster
• WordPress/Magento ready — intelligent caching, clean URLs
• Single binary — just download and run

The Numbers

Performance Comparison:
🚀 VeloServe (SAPI Mode)

~10,000 requests/sec ~1ms latency 5x faster than traditional setups

⚡ Nginx + PHP-FPM (traditional)

~2,000 requests/sec ~10ms latency Industry standard baseline

🐌 VeloServe (CGI Mode)

~500 requests/sec ~50ms latency Compatibility mode, works everywhere

The Development Journey

Day 1: Core HTTP Server

We started with the basics — a Tokio-based async HTTP server using Hyper. The initial commit was just serving static files with proper MIME types, ETag headers, and conditional requests.

I kept the Hyper migration guide open in another tab the entire time. Hyper 1.0 changed a lot, and Claude's training data didn't always have the latest patterns. Always verify.

Day 2: PHP Integration

This is where it got interesting. We implemented PHP execution in two ways:

CGI Mode — spawn php-cgi for each request, pass environment variables, pipe POST data through stdin

SAPI Mode — use Rust FFI to link against libphp.so and execute PHP in-process

The FFI work was tricky. We had to:

• Create a build.rs that detects PHP installation via php-config
• Write FFI bindings for php_embed_init(), php_execute_script(), etc.
• Handle the PHP lifecycle correctly

I spent hours in the PHP Internals Book and the Rust FFI guide making sure we weren't going to cause memory leaks or segfaults.

Day 3: WordPress Demo & Deployment

The ultimate test — can it run WordPress? We set up:

• WordPress with SQLite (no MySQL needed for demos)
• Automatic URL detection for cloud environments
• One-click deployment on Ona.com

When I saw the WordPress installation wizard load through VeloServe with ~1ms PHP execution time, I knew we had something special.

Try It Yourself

One-Line Install

curl -sSL https://veloserve.io/install.sh | bash  

Quick Test

mkdir -p /tmp/mysite  
echo '<?php phpinfo();' > /tmp/mysite/index.php  
veloserve start --root /tmp/mysite --listen 0.0.0.0:8080  

Visit http://localhost:8080 and you'll see PHP running through VeloServe.

Also a lot of useful CLI commands you can find on our Documentation pages or in Readme on Github repo:
https://github.com/veloserve/veloserve?tab=readme-ov-file#cli-tool

You can start ready-made WordPress demo:
https://github.com/veloserve/veloserve?tab=readme-ov-file#wordpress-demo-features

Try in the Cloud (No Install)

Don't want to install anything? Try it instantly:

What I Learned

1. AI is a Force Multiplier, Not a Replacement

Claude helped me write code 10x faster, but I still needed to understand what the code was doing. When we hit a Windows build issue with Unix-only signals, I knew immediately how to fix it with #[cfg(unix)] because I understood Rust's conditional compilation.

2. Always Verify Against Official Docs

AI models are trained on data that can be outdated. The Rust ecosystem moves fast. Always have the docs open:

• docs.rs for crate documentation
• The Rust Book for language features
• Official project documentation for frameworks

3. Cloud Development Environments Are Game-Changers

Working through Cursor's Remote SSH to Ona.com meant:

• Consistent environment across devices
• No "works on my machine" problems
• Easy to share and reproduce
• Powerful cloud hardware for compilation

4. Ship Early, Iterate Fast

I went from zero to a working web server with WordPress support in one day. It's not perfect — the SAPI mode needs more work, we need better error handling, and there's always more optimization to do. But it works, it's open source, and the community can help improve it.

Resources

• Website: veloserve.io
• GitHub: github.com/veloserve/veloserve
• Documentation: github.com/veloserve/veloserve/tree/main/docs
• Configuration Reference: docs/configuration.md
• Environment Variables: docs/environment-variables.md

The Stack

For those curious about the exact setup:

• IDE: Cursor Pro with Remote SSH
• AI: Claude Opus 4.5 (via Cursor)
• Cloud Environment: Ona.com (Gitpod successor)
• Language: Rust 1.75+
• Key Crates: Tokio, Hyper, tokio-rustls
• Website Hosting: Vercel
• Domain: veloserve.io

What's Next

VeloServe 1.0.0 is just the beginning. On the roadmap:

• Complete SAPI mode implementation
• FastCGI protocol support
• HTTP/3 (QUIC)
• Built-in Let's Encrypt
• Configuration hot-reload
• Prometheus metrics

Development Roadmap is here:
https://github.com/veloserve/veloserve?tab=readme-ov-file#%EF%B8%8F-development-roadmap

If you're interested in contributing or just want to try it out, head to veloserve.io and give it a spin.

And if you build something cool with it, let me know on Twitter/X or open an issue on GitHub!

https://github.com/veloserve/veloserve

Good luck, happy testing!

Hello,

It's been a while since my last post. I wanted to share today something really interesting that happened to me while upgrading my Fedora OS from version 40 to 42 (latest). Back few yours ago I was heavily using Desktop called Deepin (https://www.deepin.org/en/dde/), and before switching to Plasma KDE, I have never noticed how much you depend on tools that each desktop has. This tool is deepin-editor.

https://github.com/linuxdeepin/deepin-editor

Why I like it? What is differences between that tool and others? My answer is honestly I do not know, maybe I got used to it so much that it matters to me and I want to keep it (Keeper).

So story begins I've tried to upgrade my Fedora 40, but sadly dnf upgrade gave me error related to deepin-terminal and deepin-editor :(

Running transaction check  
Transaction check succeeded.  
Running transaction test  
The downloaded packages were saved in cache until the next successful transaction.  
You can remove cached packages by executing 'dnf clean packages'.  
Error: Transaction test error:  
  file /usr/share/deepin-terminal/translations/deepin-terminal_az.qm from install of deepin-terminal-6.0.15-1.fc42.x86_64 conflicts with file from package deepin-terminal-data-6.0.14-2.fc40.noarch
  file /usr/share/deepin-terminal/translations/deepin-terminal_bo.qm from install of deepin-terminal-6.0.15-1.fc42.x86_64 conflicts with file from package deepin-terminal-data-6.0.14-2.fc40.noarch
  file /usr/share/deepin-terminal/translations/deepin-terminal_br.qm from install of deepin-terminal-6.0.15-1.fc42.x86_64 conflicts with file from package deepin-terminal-data-6.0.14-2.fc40.noarch
  file /usr/share/deepin-terminal/translations/deepin-terminal_ca.qm from install of deepin-terminal-6.0.15-1.fc42.x86_64 conflicts with file from package deepin-terminal-data-6.0.14-2.fc40.noarch
  file /usr/share/deepin-terminal/translations/deepin-terminal_cs.qm from install of deepin-terminal-6.0.15-1.fc42.x86_64 conflicts with file from package deepin-terminal-data-6.0.14-2.fc40.noarch
  file /usr/share/deepin-terminal/translations/deepin-terminal_de.qm from install of deepin-terminal-6.0.15-1.fc42.x86_64 conflicts with file from package deepin-terminal-data-6.0.14-2.fc40.noarch
  file /usr/share/deepin-terminal/translations/deepin-terminal_el.qm from install of deepin-terminal-6.0.15-1.fc42.x86_64 conflicts with file from package deepin-terminal-data-6.0.14-2.fc40.noarch
  file /usr/share/deepin-terminal/translations/deepin-terminal_es.qm from install of deepin-terminal-6.0.15-1.fc42.x86_64 conflicts with file from package deepin-terminal-data-6.0.14-2.fc40.noarch
  file /usr/share/deepin-terminal/translations/deepin-terminal_fi.qm from install of deepin-terminal-6.0.15-1.fc42.x86_64 conflicts with file from package deepin-terminal-data-6.0.14-2.fc40.noarch
  file /usr/share/deepin-terminal/translations/deepin-terminal_fr.qm from install of deepin-terminal-6.0.15-1.fc42.x86_64 conflicts with file from package deepin-terminal-data-6.0.14-2.fc40.noarch
  file /usr/share/deepin-terminal/translations/deepin-terminal_gl_ES.qm from install of deepin-terminal-6.0.15-1.fc42.x86_64 conflicts with file from package deepin-terminal-data-6.0.14-2.fc40.noarch
  file /usr/share/deepin-terminal/translations/deepin-terminal_hi_IN.qm from install of deepin-terminal-6.0.15-1.fc42.x86_64 conflicts with file from package deepin-terminal-data-6.0.14-2.fc40.noarch
  file /usr/share/deepin-terminal/translations/deepin-terminal_hr.qm from install of deepin-terminal-6.0.15-1.fc42.x86_64 conflicts with file from package deepin-terminal-data-6.0.14-2.fc40.noarch
  file /usr/share/deepin-terminal/translations/deepin-terminal_hu.qm from install of deepin-terminal-6.0.15-1.fc42.x86_64 conflicts with file from package deepin-terminal-data-6.0.14-2.fc40.noarch
  file /usr/share/deepin-terminal/translations/deepin-terminal_id.qm from install of deepin-terminal-6.0.15-1.fc42.x86_64 conflicts with file from package deepin-terminal-data-6.0.14-2.fc40.noarch
  file /usr/share/deepin-terminal/translations/deepin-terminal_it.qm from install of deepin-terminal-6.0.15-1.fc42.x86_64 conflicts with file from package deepin-terminal-data-6.0.14-2.fc40.noarch
  file /usr/share/deepin-terminal/translations/deepin-terminal_ko.qm from install of deepin-terminal-6.0.15-1.fc42.x86_64 conflicts with file from package deepin-terminal-data-6.0.14-2.fc40.noarch
  file /usr/share/deepin-terminal/translations/deepin-terminal_ms.qm from install of deepin-terminal-6.0.15-1.fc42.x86_64 conflicts with file from package deepin-terminal-data-6.0.14-2.fc40.noarch
  file /usr/share/deepin-terminal/translations/deepin-terminal_nl.qm from install of deepin-terminal-6.0.15-1.fc42.x86_64 conflicts with file from package deepin-terminal-data-6.0.14-2.fc40.noarch
  file /usr/share/deepin-terminal/translations/deepin-terminal_pl.qm from install of deepin-terminal-6.0.15-1.fc42.x86_64 conflicts with file from package deepin-terminal-data-6.0.14-2.fc40.noarch
  file /usr/share/deepin-terminal/translations/deepin-terminal_pt.qm from install of deepin-terminal-6.0.15-1.fc42.x86_64 conflicts with file from package deepin-terminal-data-6.0.14-2.fc40.noarch
  file /usr/share/deepin-terminal/translations/deepin-terminal_pt_BR.qm from install of deepin-terminal-6.0.15-1.fc42.x86_64 conflicts with file from package deepin-terminal-data-6.0.14-2.fc40.noarch
  file /usr/share/deepin-terminal/translations/deepin-terminal_ro.qm from install of deepin-terminal-6.0.15-1.fc42.x86_64 conflicts with file from package deepin-terminal-data-6.0.14-2.fc40.noarch
  file /usr/share/deepin-terminal/translations/deepin-terminal_ru.qm from install of deepin-terminal-6.0.15-1.fc42.x86_64 conflicts with file from package deepin-terminal-data-6.0.14-2.fc40.noarch
  file /usr/share/deepin-terminal/translations/deepin-terminal_sq.qm from install of deepin-terminal-6.0.15-1.fc42.x86_64 conflicts with file from package deepin-terminal-data-6.0.14-2.fc40.noarch
  file /usr/share/deepin-terminal/translations/deepin-terminal_sr.qm from install of deepin-terminal-6.0.15-1.fc42.x86_64 conflicts with file from package deepin-terminal-data-6.0.14-2.fc40.noarch
  file /usr/share/deepin-terminal/translations/deepin-terminal_tr.qm from install of deepin-terminal-6.0.15-1.fc42.x86_64 conflicts with file from package deepin-terminal-data-6.0.14-2.fc40.noarch
  file /usr/share/deepin-terminal/translations/deepin-terminal_ug.qm from install of deepin-terminal-6.0.15-1.fc42.x86_64 conflicts with file from package deepin-terminal-data-6.0.14-2.fc40.noarch
  file /usr/share/deepin-terminal/translations/deepin-terminal_uk.qm from install of deepin-terminal-6.0.15-1.fc42.x86_64 conflicts with file from package deepin-terminal-data-6.0.14-2.fc40.noarch
  file /usr/share/deepin-terminal/translations/deepin-terminal_zh_HK.qm from install of deepin-terminal-6.0.15-1.fc42.x86_64 conflicts with file from package deepin-terminal-data-6.0.14-2.fc40.noarch
  file /usr/share/deepin-terminal/translations/deepin-terminal_zh_TW.qm from install of deepin-terminal-6.0.15-1.fc42.x86_64 conflicts with file from package deepin-terminal-data-6.0.14-2.fc40.noarch

Another one after I've removed deepin-terminal-data...

So I had to:

dnf remove deepin-terminal deepin-terminal-data deepin-editor  

Yay! Error is gone, but my favorite editor is gone as well... :(

After 15-20 minutes my fresh Fedora 42 booted, I've went again to terminal:

dnf install deepin-editor  

Breakdown is:

Installed:      qt5-qtbase-5.15.16-2.fc42.x86_64  
Required by:    deepin-editor-6.5.4-1.fc42.x86_64 → qt5-qtbase = 5.15.15 ❌  

Oh boy, so qt5 went to 5.15.16-2 in Fedora 42, but my poor old deepin-editor still requires 5.15.15

Started to explore if there is any "fix" or "workaround" on fedora pages, but nothing reported so far. Probably use of this tool is so low that nobody cares to be honest :)

I've decided to go my way... as always... going to rebuild RPM, fix this and have my own personal Deepin Editor tweaked.

1) Get the source RPM:

dnf download --source deepin-editor  

2) Install RPM build tools:

sudo dnf install rpmdevtools  
rpmdev-setuptree  

3) Extract and edit the .spec:

rpm -ivh deepin-editor*.src.rpm  
cd ~/rpmbuild/SPECS  

Now, I now proper way would be to extract ~/rpmbuild/SOURCES tar archive, cp directory to something like .orig, edit original directory and then just create a patch using diff, for example creating a patch:

cd ~/rpmbuild/SOURCES  
diff -uNr deepin-editor-6.5.4.orig deepin-editor-6.5.4 > fix-qstring-null.patch  

Since initially when you run following command without any tweak, the core problem why this all is happening is that qt5 5.15 this was used:

QString::null  

Now, it has to be changed to the following:

QString()  

That's it! Damn.... that must be an easy fix right? Well, let's continue. I've decided to just untar archive within ~/rpmbuild/SOURCES directory, make an edits... yep cowboy... and good to go...

cd ~/rpmbuild/SOURCES  
grep -R "QString::null"  

You will see which files needs to be edited, I will list them here:

src/controls/tabbar.h (two places)  
src/common/settings.h  
tests/src/common/ut_utils.cpp  

Open each file and change everywhere you find QString::null to Qstring()

Navigate to the /root/rpmbuild/SOURCES/deepin-editor-6.5.4 directory and let's modify CMakeLists.txt file:

Find following:

set(CMAKE_CXX_STANDARD 11)  

and change it to C++17

set(CMAKE_CXX_STANDARD 17)  
set(CMAKE_CXX_STANDARD_REQUIRED ON)  
set(CMAKE_CXX_EXTENSIONS ON)  

After editing the file, re-create the archive and re-run:

rpmbuild -ba SPECS/deepin-editor.spec  

You will see message that build completes, and now it is ready to install your freshly customized package:

rpm -ivh /root/rpmbuild/RPMS/x86_64/deepin-editor-6.5.4-1.fc42.x86_64.rpm  

Job done! Time to test!

Great work, now you have ideas how to customized ready made RPM packages. Good luck! Hope this helps.

Hello,

It's been a while since my last post. Today I would like to show how to setup Kubernetes cluster on AWS using their Elastic Kubernetes Service. In order to do that I have selected eksctl tool (https://eksctl.io/).

You can setup managed and non-managed type of cluster. Just to mention that if you select non-Managed you are going to be able to utilize Ubuntu or any other operating system, while AWS EKS Managed services are explicitly tied to AMIs based on AmazonLinux2023 OS.

What is Longhorn filesystem?

Longhorn is an open-source, lightweight, and highly available distributed block storage solution for Kubernetes. It provides persistent storage using containers and microservices to manage storage volumes across Kubernetes clusters. Longhorn ensures data replication across multiple nodes, making it resilient to hardware failures and node outages. It features incremental snapshots, backups, and easy volume recovery, making it ideal for applications requiring reliable and scalable storage in a Kubernetes environment.

Prerequisites:

- eksctl tool installed (https://eksctl.io)
- aws cli tool (https://aws.amazon.com/cli/)
- helm tool (https://helm.sh/docs/intro/install/)
- kubectl tool (https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/)
- longhorn cli tool (https://longhorn.io/docs/1.7.0/deploy/install/#using-the-longhorn-command-line-tool)

Let's get started. First we will spin up fresh AWS EKS cluster. In my example setup I did a demo cluster with 8 worker nodes, where each Area Zone had 4.

Example of cluster.yaml file for eksctl tool is here:

apiVersion: eksctl.io/v1alpha5  
kind: ClusterConfig

metadata:  
  name: sqlscale
  region: us-east-1

availabilityZones: ["us-east-1a", "us-east-1b"]

addons:  
  - name: vpc-cni
    version: latest
  - name: coredns
    version: latest
  - name: kube-proxy
    version: latest

managedNodeGroups:  
  - name: sqlng-1
    instanceType: t2.medium
    instanceName: sqlscale-worker-1
    desiredCapacity: 4
    iam:
      attachPolicyARNs:
        - arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy
        - arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy
        - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly
        - arn:aws:iam::aws:policy/ElasticLoadBalancingFullAccess
        - arn:aws:iam::aws:policy/AmazonSSMFullAccess
      withAddonPolicies:
        imageBuilder: true
        autoScaler: true
        externalDNS: true
        certManager: true
        appMesh: true
        appMeshPreview: true
        ebs: true
        fsx: true
        efs: true
        awsLoadBalancerController: true
        xRay: true
        cloudWatch: true
    volumeSize: 50
    volumeType: gp3
    ssh:
      allow: true # will use ~/.ssh/id_rsa.pub as the default ssh key
    privateNetworking: true
    labels: {role: worker}
    tags:
      nodegroup-role: worker
    preBootstrapCommands:
      # Install SSM Agent,NFS,Openiscsi and similar packages
      - "yum install -y amazon-ssm-agent"
      - "yum install nfs-utils -y"
      - "yum --setopt=tsflags=noscripts install iscsi-initiator-utils -y"
      - "yum install curl -y"
      - 'echo "InitiatorName=$(/sbin/iscsi-iname)" > /etc/iscsi/initiatorname.iscsi'
      - "systemctl enable iscsid"
      - "systemctl start iscsid"
      - "systemctl enable amazon-ssm-agent"
      - "systemctl start amazon-ssm-agent"
      # allow docker registries to be deployed as cluster service
      - "sed '2i \"insecure-registries\": [\"172.20.0.0/16\",\"10.100.0.0/16\"],'  /etc/docker/daemon.json"
      - "systemctl restart docker"
  - name: sqlng-2
    instanceType: t2.medium
    instanceName: sqlscale-worker-2
    desiredCapacity: 4
    iam:
      attachPolicyARNs:
        - arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy
        - arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy
        - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly
        - arn:aws:iam::aws:policy/ElasticLoadBalancingFullAccess
        - arn:aws:iam::aws:policy/AmazonSSMFullAccess
      withAddonPolicies:
        imageBuilder: true
        autoScaler: true
        externalDNS: true
        certManager: true
        appMesh: true
        appMeshPreview: true
        ebs: true
        fsx: true
        efs: true
        awsLoadBalancerController: true
        xRay: true
        cloudWatch: true
    volumeSize: 50
    volumeType: gp3
    ssh:
      allow: true # will use ~/.ssh/id_rsa.pub as the default ssh key
    privateNetworking: true
    labels: {role: worker}
    tags:
      nodegroup-role: worker
    preBootstrapCommands:
      # Install SSM Agent,NFS,Openiscsi and similar packages
      - "yum install -y amazon-ssm-agent"
      - "yum install nfs-utils -y"
      - "yum --setopt=tsflags=noscripts install iscsi-initiator-utils -y"
      - "yum install curl -y"
      - 'echo "InitiatorName=$(/sbin/iscsi-iname)" > /etc/iscsi/initiatorname.iscsi'
      - "systemctl enable iscsid"
      - "systemctl start iscsid"
      - "systemctl enable amazon-ssm-agent"
      - "systemctl start amazon-ssm-agent"
      # allow docker registries to be deployed as cluster service
      - "sed '2i \"insecure-registries\": [\"172.20.0.0/16\",\"10.100.0.0/16\"],'  /etc/docker/daemon.json"
      - "systemctl restart docker"

** Feel free to edit various settings. This example has what is required to install and setup AWS EKS cluster with Longhorn filesystem successfully.

You can check following page for other possible options to integrate into your cluster.yaml file:
https://eksctl.io/usage/creating-and-managing-clusters/

Export your AWS Secret and Access keys in Shell or configure it with
aws configure

Push to following command to provision your AWS EKS cluster:

eksctl create cluster -f cluster.yaml  

It takes a while to complete, so sit back and monitor your terminal screen output.

Once AWS EKS Cluster is ready and you have tested kubectl we are going to use longhornctl tool to install and set up the preflight dependencies before installing Longhorn.

./longhornctl --kube-config='/home/nemke/.kube/config' install preflight

Output:

You may check if all software is installed using following command:

./longhornctl --kube-config='/home/nemke/.kube/config' check preflight

Output:

Words of wisdom here.... Well, if you plan to use HPA (Horizontal Pod Autoscaler) with AWS EKS Autoscaling groups, I advise that you make sure that all packages are getting installed within cluster.yaml file. Why? Well, eksctl tool is using CloudFormattion so any new node added will get packages installed before new node joins your fleet.

Next we will install Longhorn filesystem and configure it.

Add the Longhorn Helm repository:

helm repo add longhorn https://charts.longhorn.io  

Fetch the latest charts from the repository:

helm repo update  

Install Longhorn in the longhorn-system namespace:

helm install longhorn longhorn/longhorn --namespace longhorn-system --create-namespace --version 1.7.0  

Output:

To confirm that the deployment succeeded, run:

kubectl -n longhorn-system get pod  

One of the super neat features I've seen with Longhorn is its UI. To enable access to the Longhorn UI, you will need to set up an Ingress controller. Authentication to the Longhorn UI is not enabled by default. For information on creating an NGINX Ingress controller with basic authentication, refer to the following link: https://longhorn.io/docs/1.7.0/deploy/accessing-the-ui/longhorn-ingress

If you install Longhorn on a Kubernetes cluster with kubectl or Helm, you will need to create an Ingress to allow external traffic to reach the Longhorn UI.

Authentication is not enabled by default for kubectl and Helm installations. In these steps, you’ll learn how to create an Ingress with basic authentication using annotations for the nginx ingress controller.

1) Create a basic auth file auth. It’s important the file generated is named auth (actually - that the secret has a key data.auth), otherwise the Ingress returns a 503.

USER=<USERNAME_HERE>; PASSWORD=<PASSWORD_HERE>; echo "${USER}:$(openssl passwd -stdin -apr1 <<< ${PASSWORD})" >> auth  

2) Create a secret:

kubectl -n longhorn-system create secret generic basic-auth --from-file=auth  

3) Create an Ingress manifest longhorn-ingress.yml:

apiVersion: networking.k8s.io/v1  
kind: Ingress  
metadata:  
  name: longhorn-ingress
  namespace: longhorn-system
  annotations:
    # type of authentication
    nginx.ingress.kubernetes.io/auth-type: basic
    # prevent the controller from redirecting (308) to HTTPS
    nginx.ingress.kubernetes.io/ssl-redirect: 'false'
    # name of the secret that contains the user/password definitions
    nginx.ingress.kubernetes.io/auth-secret: basic-auth
    # message to display with an appropriate context why the authentication is required
    nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required '
    # custom max body size for file uploading like backing image uploading
    nginx.ingress.kubernetes.io/proxy-body-size: 10000m
spec:  
  ingressClassName: nginx
  rules:
  - host: longhorn.example.com
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: longhorn-frontend
            port:
              number: 80

** As you may notice I've used longhorn.example.com as a URL to login to the Longhorn UI from my browser. You may adjust it to real domain name with DNS, but it is not required because we can use /etc/hosts mechanism to access it.

4) Create the Ingress:

kubectl -n longhorn-system apply -f longhorn-ingress.yml  

You will need to create an ELB (Elastic Load Balancer) to expose the nginx Ingress controller to the Internet. Additional costs may apply.

5) Let's create ELB using following command:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/aws/deploy.yaml  

** You may use Helm to deploy this or adjust ELB settings, but for the purpose of this Demo and UI I've used default values.

You can now navigate to AWS Console --> EC2 --> Load Balancer area, and you will notice that Load Balancer is being provisioned:

Grab two IP addresses assigned to your new Load Balancer, and adjust your /etc/hosts file with the following data:

52.86.18.142 longhorn.example.com  
44.208.155.179 longhorn.example.com  

You can dig DNS endpoint to get them.

Navigate to the following URL to get to the Longhorn UI:
https://longhorn.example.com
(ignore SSL errors of course)

Enter username and password that you have defined in the Create a basic auth file auth step.

Output:

Welcome to Longhorn UI. No errors, and as you can see 8 nodes in the group (4 workers us-east-1a and 4 workers in us-east-1b). Feel free to explore https://longhorn.io/docs/1.7.0/ docs pages, but the most basic is to visit Settings --> General area and adjust what is default number of replicas you wish to keep for each Volume that is created followed by PV/PVC from your Kubernetes deployment.

That's it for now. In next articles, we will use this setup to provision MariaDB server with Master/Slave replicas,and then MariaDB server "Galera" with Master/Master replicas.

Hope this article helps! Good luck!

I often see ElasticSearch booted into single-node mode and show cluster health in a "Yellow" mode. I want to explain why that is happening today and what we can do to prevent/fix that, even in single-node mode.

Elasticsearch is a highly scalable, open-source search and analytics engine built on top of Apache Lucene. It is designed for horizontal scalability and near real-time search and analytics capabilities. Elasticsearch is commonly used for various use cases, including log and event data analysis, full-text search, and business analytics. Its powerful querying capabilities and distributed nature make it a preferred choice for handling large datasets and complex search operations.

Cluster Health:

In Elasticsearch, data is distributed across multiple nodes in a cluster to ensure high availability, fault tolerance, and load balancing. Cluster health status indicates the overall well-being and operational status of an Elasticsearch cluster. The cluster health can be one of three states:

Green: All primary and replica shards are allocated. This is the optimal state, indicating that the cluster is fully functional with redundancy.

Yellow: All primary shards are allocated, but some or all replica shards are not. While the cluster is operational, it lacks redundancy, which means it is vulnerable to data loss if a node fails.

Red: One or more primary shards are unassigned, leading to potential data loss and unavailability of some data.

Maintaining a green cluster health status is crucial because it ensures that your data is not only available but also redundant, providing resilience against node failures and ensuring high availability of your services. A healthy cluster also optimizes performance and reliability, which are essential for search and analytics operations.

Now, guess you have a Magento 2 installation, you reindex but when you execute following command your Cluster state (yes, even in a single node) is always Yellow.

Check Current Cluster Health

curl -X GET "localhost:9200/_cluster/health?pretty"  

We can Analyze Indices and Shards using following command

curl -X GET "localhost:9200/_cat/indices/?pretty"  

To bring your Elasticsearch cluster to a green state from the current yellow state, you need to address the issue of unassigned shards. In your case, this typically occurs because your cluster has only one data node, which means there is no other node to allocate replica shards to.

Steps to Bring the Cluster to a Green State

Idea 1:

• Add More Data Nodes:
  The best way to achieve a green state is to add more data nodes to the cluster to properly allocate replicas. Here's a quick guide on how to add a node:

1) Install Elasticsearch on another server.
2) Configure the new node to join the existing cluster by setting the same cluster.name in the elasticsearch.yml file.
3) Set the node.name to a unique name.
4) Ensure network settings (network.host, discovery.seed_hosts, etc.) are correctly configured to allow the nodes to communicate.
5) Start Elasticsearch on the new server.

Idea 2 (and stay on the single node):

• Adjust Replica Settings: If adding more nodes is not feasible, you can adjust the replica settings to 0 for indices with unassigned replicas. This will bring the cluster to a green state, but you will lose redundancy. Here's how you can do it:

# Set the number of replicas to 0 for a specific index
curl -X PUT "localhost:9200/<index_name>/_settings" -H 'Content-Type: application/json' -d'  
{
  "index": {
    "number_of_replicas": 0
  }
}'

The next step is to verify Cluster Health

After making these changes, verify the cluster health:

curl -X GET "localhost:9200/_cluster/health?pretty"  

Boom! It's in the green state:

{"acknowledged":true}[root@server ~]# curl -X GET "localhost:9200/_cluster/health?pretty"
{
  "cluster_name": "magento",
  "status": "green",
  "timed_out": false,
  "number_of_nodes": 1,
  "number_of_data_nodes": 1,
  "active_primary_shards": 7,
  "active_shards": 7,
  "relocating_shards": 0,
  "initializing_shards": 0,
  "unassigned_shards": 0,
  "delayed_unassigned_shards": 0,
  "number_of_pending_tasks": 0,
  "number_of_in_flight_fetch": 0,
  "task_max_waiting_in_queue_millis": 0,
  "active_shards_percent_as_number": 100.0
}
[root@server ~]# curl -X GET "localhost:9200/_cat/indices/?pretty"
green open .geoip_databases                                          jMYDTc1NRZuQDZ9tGtyGSQ 1 0   34    34 31.3mb 31.3mb  
green open xyz_production_22052024_amasty_elastic_popup_data_1_v2 VoF-bkUDQEef-HeFzS-4og 1 0  196     0 91.4kb 91.4kb  
green open xyz_production_22052024_product_1_v2                   e7xmqI3XTgWprQB4EWuD8g 1 0 4352 64006 36.1mb 36.1mb  

By following these steps, you should be able to bring your Elasticsearch cluster to a green state, ensuring that all shards are allocated and there are no unassigned shards. Now, index names can change, so I suggest extending core functionalities to add another function, which would be helpful each time indexing is running.

I hope this article helps. Good luck!

Hello,

I have been exploring options lately to scale MySQL and tested this solution. In my tests, I am using small db.t2.medium AWS RDS cluster with Write and Read replica.

For test purposes following components were used:
- AWS account setup of RDS with public access with IP-only limitation.
- Magento 2 gitpod instance (https://github.com/nemke82/magento2gitpod) started.
- Modified m2-install.sh file to connect to Remote AWS RDS environment and install fresh Magento 2 install with Performance fixtures data (small).
- ProxySQL installed on the Gitpod environment and configured

Before you start, you have to create a test database. In my example:

mysql -u nemanja -h nemanja-instance-1.cusgvuriflp3.us-east-1.rds.amazonaws.com -pRandomPassword123 -e 'create database nemanja;'  

Then we can add privileges to that database:

GRANT ALL PRIVILEGES ON nemanja.* TO 'nemanja'@'%';  
FLUSH PRIVILEGES;  
exit;  

Modify m2-install.sh (if you are testing on the Magento 2 Gitpod platform) file with the new MySQL hostname endpoint, username, and password and execute the installation.

Once the installation of fresh Magento 2 is ready and data is in your AWS RDS instance, we will proceed with ProxySQL configuration.

To set up ProxySQL to split SELECT queries between an AWS RDS master instance and its read replicas, follow these steps:

Install ProxySQL
Install ProxySQL on an EC2 instance or a server within your VPC, so it has access to your RDS instances.

URL: https://proxysql.com/documentation/installing-proxysql/

Configure ProxySQL
Edit the ProxySQL configuration file, usually located at /etc/proxysql.cnf, and configure it according to your needs. An example configuration is shown below:

datadir="/var/lib/proxysql"  
admin_variables=  
{
  admin_credentials="admin:admin"
  mysql_ifaces="0.0.0.0:6032"
}
mysql_variables=  
{
  threads=4
  max_connections=2048
  default_query_delay=0
  default_query_timeout=36000000
  have_compress=true
  poll_timeout=2000
  interfaces="0.0.0.0:6033"
  default_schema="information_schema"
  stacksize=1048576
  server_version="5.7.27"
  connect_timeout_server=3000
  monitor_history=600000
  monitor_connect_interval=60000
  monitor_ping_interval=10000
  ping_interval_server_msec=10000
  ping_timeout_server=200
  commands_stats=true
  sessions_sort=true
}

Start the ProxySQL service:

sudo systemctl start proxysql  

Next task is to Configure RDS instances in ProxySQL.

Let's access the ProxySQL admin interface:

mysql -u admin -p -h 127.0.0.1 -P 6032 --prompt='ProxySQLAdmin> '  

Default password is "admin" for Fresh ProxySQL installation.

Add your RDS master and read replica instances:

-- Add master instance
INSERT INTO mysql_servers(hostgroup_id, hostname, port)  
VALUES (0, 'your_master_instance_endpoint', 3306);

-- Add read replica instance
INSERT INTO mysql_servers(hostgroup_id, hostname, port)  
VALUES (1, 'your_read_replica_instance_endpoint', 3306);

LOAD MYSQL SERVERS TO RUNTIME;  
SAVE MYSQL SERVERS TO DISK;  

Here we start with one master and 1 read replica endpoint. We can scale later and add more easily.

Set up query rules

The next step is to create rules to route SELECT queries to the read replica:

INSERT INTO mysql_query_rules (rule_id, active, match_pattern, destination_hostgroup, apply)  
VALUES (1, 1, '^SELECT.*FOR UPDATE$', 0, 1),  
       (2, 1, '^SELECT', 1, 1);             

LOAD MYSQL QUERY RULES TO RUNTIME;  
SAVE MYSQL QUERY RULES TO DISK;  

-- Send 'SELECT ... FOR UPDATE' to the master
-- Send other SELECTs to the read replica
You can extend this with anything you think could benefit route SELECT queries to the read replica.

Configure user credentials

Add the database user credentials to ProxySQL:

INSERT INTO mysql_users(username, password, default_hostgroup)  
VALUES ('your_username', 'your_password', 0);

LOAD MYSQL USERS TO RUNTIME;  
SAVE MYSQL USERS TO DISK;  

We will use the same username and password we used to define when creating the AWS RDS cluster. I advise using anything except the "admin" username, as that is the default username for ProxySQL, which we could change later, but is not crucial in the current proof-of-concept demo.

Monitor and adjust
Monitor ProxySQL's performance and adjust settings as needed. You can add more read replicas and change the load balancing settings for better performance.

Start by adjusting the env.php file. In the 'host' field, enter:

127.0.0.1:6033  

We could experiment here by adding Socket instead of a loopback which I think should optimize this area better.

Next, adjust the 'username' and 'password' fields with the one you created above.

Now, your application should connect to ProxySQL on port 6033 instead of directly connecting to the RDS instances. ProxySQL will route SELECT queries to the read replica and other write queries to the master instance.

-- Horizontally scaling Read replicas --
To add more read replicas and adjust the load balancing settings for better performance, follow these steps:

Add more read replicas in AWS RDS
First, create additional read replicas of your master RDS instance from the AWS Management Console or the AWS CLI.

Add read replicas to ProxySQL
Connect to the ProxySQL admin interface:

mysql -u admin -p -h 127.0.0.1 -P 6032 --prompt='ProxySQLAdmin> '  

Add the new read replica instances to ProxySQL, assigning them to the same hostgroup (1 in this example):

INSERT INTO mysql_servers(hostgroup_id, hostname, port)  
VALUES (1, 'your_additional_read_replica_instance_endpoint_1', 3306),  
       (1, 'your_additional_read_replica_instance_endpoint_2', 3306),
       ...;

LOAD MYSQL SERVERS TO RUNTIME;  
SAVE MYSQL SERVERS TO DISK;  

-- Add additional read replica instances

Configure load balancing settings
By default, ProxySQL uses the round-robin algorithm to balance the load among the read replicas. You can adjust the weight of each server to control the traffic distribution. A higher weight means more traffic will be sent to that server.

To modify the weight of a read replica, update the weight column in the mysql_servers table:

UPDATE mysql_servers SET weight = new_weight  
WHERE hostgroup_id = 1 AND hostname = 'your_read_replica_instance_endpoint';

LOAD MYSQL SERVERS TO RUNTIME;  
SAVE MYSQL SERVERS TO DISK;  

Here we can say for readreplicainstance_endpoint. We will put 50, then next one, we can put 50 if two read replicas are added. Overall that makes them split traffic even. Monitor the performance of your read replicas and adjust their weights accordingly. You can use the ProxySQL stats schema to gather information about traffic distribution and query performance.

For example, to view the traffic distribution among the read replicas, you can run:

select * from stats_mysql_connection_pool;  

You can see several queries sent and other useful data:

Adjust the read replica weights based on the performance metrics and your application's requirements. Continue to monitor and fine-tune the load balancing settings to achieve optimal performance.

Remember that adding more read replicas may increase costs but can help distribute read traffic and improve performance for read-heavy workloads. Ensure you monitor your instances' resource usage to balance performance and price. All the above can be partially or fully automated with a systemd or cron task checking the state of the AWS RDS server and then scaling up replicas based on selected patterns.

We can also enable ProxySQL UI and watch stats in the browser.
Ref article: https://proxysql.com/documentation/http-web-server/

Connect to the ProxySQL admin interface:

mysql -u admin -p -h 127.0.0.1 -P 6032 --prompt='ProxySQLAdmin> '  

Then:

SET admin-web_enabled='true';  
LOAD ADMIN VARIABLES TO RUNTIME;  

I usually use SSH tunnel to connect and get to the UI, for example:

ssh -p <SSHport> -L 6080:127.0.0.1:6080 <SSHuser>@<SSHhost>  

Then you can open up your favorite browser and visit https://127.0.0.1:6080
username and password by default are admin:admin

Where are problems here?

A typical issue with Indexer is the following:

Product Price index process error during indexation process:  
SQLSTATE[Y0000]: <<Unknown error>>: 9006 ProxySQL Error: connection is locked to hostgroup 0 but trying to reach hostgroup 1, query was: SELECT `i`.`entity_id`, `o`.`option_id` FROM `catalog_product_index_price_temp` AS `i`  
 INNER JOIN `catalog_product_entity` AS `e` ON e.entity_id = i.entity_id
 INNER JOIN `catalog_product_option` AS `o` ON o.product_id = e.entity_id
Catalog Search index process error during indexation process:  
SQLSTATE[Y0000]: <<Unknown error>>: 9006 ProxySQL Error: connection is locked to hostgroup 0 but trying to reach hostgroup 1, query was: SELECT `indexer_state`.* FROM `indexer_state` WHERE (`indexer_state`.`indexer_id`='catalogsearch_fulltext')  

To fix:
As is explained in the documentation

https://proxysql.com/documentation/global-variables/mysql-variables/#mysql-setquerylockonhostgroup

So to resolve this issue on the single primary server on Group replication, try to do this.

Log into ProxySql admin monitor.

set mysql-set_query_lock_on_hostgroup=0;  
load mysql variables to runtime;  
save mysql variables to disk;  

Not mandatory, but the ProxySql service can be restarted to check if variables are saved on disk. Every setting here was done with "hot-reload". The advantage of ProxySQL service is that we can use proxysql.cnf file and add them as default instead.

Stress testing this just phase one. I will update you with additional how time goes on. Using the tool mysqlslap
(https://dev.mysql.com/doc/refman/8.0/en/mysqlslap.html), I've tried to stress/load test this with 100 concurrent users sending some generic SELECT query to see how READS are spread across the board. It was to select all Quote Items:

gitpod /workspace/magento2gitpod $ mysqlslap -unemanja -P6033 -p -h127.0.0.1  --concurrency=100 --iterations=20 --create-schema=nemanja --query="SELECT * from quote_item" --verbose  
Enter password:  

Benchmark
The average number of seconds to run all queries: 0.369 seconds
Minimum number of seconds to run all queries: 0.218 seconds
Maximum number of seconds to run all queries: 1.144 seconds
Number of clients running queries: 100
The average number of queries per client: 1

Next, Benchmark using the same query was a lot faster since the query entered Query Cache, which ProxySQL provides by default even if AWS RDS is 8.0.26 used.

Benchmark
The average number of seconds to run all queries: 0.292 seconds
Minimum number of seconds to run all queries: 0.184 seconds
Maximum number of seconds to run all queries: 0.494 seconds
Number of clients running queries: 100
The average number of queries per client: 1

You can load separate terminals or screens (monitor) and watch how SELECT queries from the same Load/Stress test are spread across two Read replicas that I've created in this Concept Proof demo:

watch -n 1 "mysql -u admin -padmin -h 127.0.0.1 -P 6032 -e 'select * from stats_mysql_connection_pool;'"  

I hope this article was helpful. Good luck!

Hello,

Recently, I discovered the issue when trying to start the PWA Studio development environment.

Invalid host header

command used: yarn watch

To fix this, you can add the following to the node_modules/@magento/pwa-buildpack/lib/WebpackTools/PWADevServer.js

disableHostCheck: true,  

Save the file, start yarn watch again and enjoy development that can refresh content without compiling.

I hope this helps.

Hello,

By default, Magento 2 offers to debug Varnish if pages are in MISS or HIT mode when Developer mode is enabled. You can debug the store in two ways in the Production mode.

1) varnishlog
varnishlog is a CLI tool written to read the output from the Varnish service in format order. You can print output on the terminal, parse it to other formats, stream outside of the container or service, or similar. For more details about the varnishlog tool, you can read the following https://varnish-cache.org/docs/trunk/reference/varnishlog.html address.

To limit output only to your IP address, we can use the following CLI command:

varnishlog -q "ReqHeader eq 'X-Real-IP: 178.223.40.243'"  

Note: Update your real IP address. You can use X-Real-IP or X-Forward-For depending on your Upstream and Webserver settings.

The standard output has a lot of information, but mostly it starts with:

<< Request  >>  

Then it has a timestamp, ReqStart, ReqMethod (GET, POST...), ReqURL, and ReqProtocol, but the longest part is ReqHeader and RespHeader section. varnishlog is not the tool you want to collect logs to store. However, it’s the perfect choice to obtain debugging data thanks to the indecent amount of data it produces.

The one I find important to debug the Varnish cache is:

VCL_call  

It can end up as DELIVER (not cached) or HIT (cached). You can read Varnish documentation pages Varnish Output on how many other options can be used. Typically followed by VCL_call Deliver is RespHeader Age: 0

2) If you find this hard and not efficient enough (yeah, I know the browser/inspect header sounds better), we can do the following to debug and make it easier.

• Under sub vcl_deliver { section comment few lines.

 //if (resp.http.X-Magento-Debug) {
        if (resp.http.x-varnish ~ " ") {
            set resp.http.X-Magento-Cache-Debug = "HIT";
        } else {
            set resp.http.X-Magento-Cache-Debug = "MISS";
        }
    //} else {
       // # unset resp.http.Age;
    //}

Note: by default, it's without //

Save the VCL file, test, and reload your Varnish cache. You will see there is a new RespHeader added on page load.

What is x-magento-cache-debug you can read here:
https://experienceleague.adobe.com/docs/commerce-operations/configuration-guide/cache/varnish/config-varnish-final.html

That's it! Now you follow this simple trick, leave your store in the Production mode, and still debug your Varnish cache if pages are HIT or MISS.

Good luck!

Hello,

Today I would like to explain a simple trick of how you can search and analyze any data stored in Redis and then export it in JSON format to filter better.

You can get the dump if you export data on the production node/server.rdb file from Redis root folder (CONFIG GET DIR), or if Persistance mode is not enabled (something ephemeral), you can extract data using the https://github.com/r043v/rdd tool.

RDB is a dump. You can call save to force an Rdb. It will be stored in the DB filename setting you have, or dump.rdb in the current working directory if that setting is missing.

More Info: http://redis.io/topics/persistence

Now that rdb dump is ready on your local computer or any other device/server, you can start by installing Redis-Commander.

What is Redis Commander?
Redis-Commander is a node.js web application used to view, edit, and manage a Redis Database.

More Info: http://joeferner.github.io/redis-commander/

Super simple to get going with NPM:

npm install -g redis-commander  

Start it with:

redis-commander  

Then point your browser to the local computer's address in the console. By default, access with browser at http://127.0.0.1:8081 address.

If you are using any Remote IDE environment like Gitpod (https://gitpod.io), you can get the address exposed when Redis-commander service starts.

Make sure you have Redis installed. I will not write about how to install the Redis server with cli tools but make sure it started on your local computer or remote device/server.

To check if Redis Commander works, load UI, and you must see it connected to your Redis server.

If you wish to start the server with the Redis dump you exported from your server, yes, you can do it stop this one and start the server again with the following command:

/usr/bin/redis-server --port 6379 --dbfilename backup_of_master.rdb

** my redis-server is at /usr/bin path, but you can type whereis redis-server to see where it is installed.

You can start analyzing data. There is also a helper who can help you with certain commands. One of the best functionalities here is that we can now export data in JSON format and analyze values much better using sublime or any other popular editor or even Online using https://jsoneditoronline.org/classic/index.html#left=local.sugeve website. Neat!

2nd popular application to analyze data from Redis is https://resp.app/

Good luck, happy troubleshooting!

Hello,

Today I want to show how you can use your local VS Code IDE editor to manage Remote Kubernetes Clusters and connect to running containers, browse local files/folders, and works with MySQL, Redis, ElasticSearch, Debug, and a lot more!

I will skip the part where I wrote one of the possible methods how to connect your kubectl tool to the Remote Kubernetes cluster running in the Cloud or Bare metal servers, and that is already described briefly in the https://nemanja.io/manage-remote-kubernetes-clusters-with-kubectl-and-lens/ article, so please open it now if you need to set up that parts on your local computer.

Please visit the following URL to set up the Remote Kubernetes cluster https://marketplace.visualstudio.com/items?itemName=ms-kubernetes-tools.vscode-kubernetes-tools address or Launch VS Code Quick Open (Ctrl+P), paste the following command, and press enter.

ext install ms-kubernetes-tools.vscode-kubernetes-tools  

Once the module is installed, you can click on the left sidebar on the icon and start exploring.

You will notice that Cluster is kubernetes-admin@kubernetes. That is the one you set up using my tutorial with the SSH Tunnel mechanism. This extension has very big documentation of what you can actually do with it from your VS Code editor. Feel free to review https://github.com/vscode-kubernetes-tools/vscode-kubernetes-tools URL. I will cover just the basics of what you need, for example, to connect/attach to a container within a namespace, and one of the best features is "Attach Visual Studio Code" which remotely connects you to the specific container in a separate VS Code window, and you can work there same as you do locally and even install extensions on that container which can be used again next time.

Press CTRL+SHIFT+P keys, then type kubernetes

Select Kubernetes: Use Namespace option and start searching your namespace name, which I assume you already know. You can also explore all namespaces (slower if you have a lot of them) and then search from that area simply by typing keywords.

Once the specific namespace is selected, we can explore everything that Kubernetes offers, like Deployments, Workloads, Secretes, and ConfigMaps. I like this because you can edit something within the YAML file, and with a single click on X, it will check, save, and Kubernetes is going to restart to the latest version. You can also schedule this change for any time that is suitable, close your browser and enjoy your day while the task is going to be scheduled in the Kubernetes.

Deployments --> Workload --> Pods you can simply SSH into any available container within POD, and you will be asked if there are multiple within a single Kubernetes pod.

From the same area right, click, you can do other things like grab logs, describe pod/container or watch it actively while troubleshooting start/stop into a separate window.

Next super feature of this extension is that you can actually "Attach" VS Code IDE server directly into any Kubernetes Pod/Container and remotely connect to it, working from your computer the same way as you do locally. I find this super useful because of many aspects, but I will repeat here few:

• Search is amazing, fast, and accurate
• You can edit any file and parse them with many useful VS Code extensions like a parser, tabnine, or similar
• Smart log extension https://marketplace.visualstudio.com/items?itemName=mbehr1.smart-log
• MySQL/ElasticSearch/Redis browser/editor https://marketplace.visualstudio.com/items?itemName=cweijan.vscode-mysql-client2

You can install anything useful that will help you develop, troubleshoot/debug or test just now with the aspect that you can work from the same place and operate with the Kubernetes cluster.

I hope this article was helpful. Happy DevOpsing :)

Hello,

Today I want to show how to use another tremendous external tool to Minify Theme's CSS files, skipping Magento 2 built-in (Core) functionality.

Minification is taking some code and using various methods to reduce its size on disk. Unlike techniques such as gzip, which preserve the original semantics of the CSS file and are therefore lossless, minification is an inherently lossy process, where values can be replaced with smaller equivalents or selectors merged, for example.

The final result of a minification step is that the resulting code will behave the same as the original file, but some parts will be altered to reduce the size as much as possible.

Combining gzip compression with minification leads to the best reduction in file size, but don't just take our word for it.

Vendor suggests trying out css-size, a module especially created to measure CSS size before & after minification.
https://npmjs.org/package/css-size

cssnano tool is one such minifier, which is written in Node.js. It's a PostCSS plugin that you can add to your build process, to ensure that the resulting stylesheet is as small as possible for a production environment.

To get started as per vendor https://cssnano.co/docs/getting-started/ page we will install cssnano and postcss CLI tools.

npm install --save-dev postcss cssnano cssnano-cli postcss-cli  

You can do it with -g for global purposes

npm install --save-dev postcss cssnano cssnano-cli postcss-cli -g  

Now that tool is installed you can type whereis command to get the path of postcss tool

whereis postcss  

Once you have done this, you will need to configure cssnano by creating a postcss.config.js file in the root of your project. This should contain cssnano as well as any other plugins that you might want for your project; as an example:

module.exports = {  
    plugins: [
        require('cssnano')({
            preset: 'default',
        }),
    ],
};

Read more about presets at https://cssnano.co/docs/presets there are tons of other options available but for Magento 2 specific I have tested with the "default" preset.

Next, we will get the THEME folder path scripted into the bash variable

THEME_FOLDER=('/workspace/magento2gitpod/pub/static/frontend/THEME/porto/en_US')  

Note: make sure you enter the correct path to your deployed theme folder in the pub/ directory

Time to push command and replace current CSS files with the minified version of them using the same file names.

find ${THEME_FOLDER[@]} \( -name '*.css' -not -name '*.min.css' -not -name '*.json' -not -name '*.js' \) -exec /usr/local/nvm/versions/node/v16.14.0/bin/postcss --replace \{} \;  

Note: You see, I've used the nvm (Node Version Manager) tool and a specific v16.14.0 version. Tool vendors claim that you can use anything after version 10 safely.

Time to measure before/after :)

npm install -g css-size  

Pick one random CSS file before running it, for example:

Auspicious results :) but okay, some files give more significant differences some do not. With the Advanced preset, you can remove comments, which is good in the production environment to save more space and reduce CSS sizes.

Here you can find detailed instructions on what CSSnano Advanced preset offers:
https://cssnano.co/docs/advanced-transforms/
https://cssnano.co/docs/what-are-optimisations/

Since the tool is pure CLI you can integrate it within any deployment CI/CD process and run it always after pushing setup:static-content:deploy command from the native Magento tool.

Happy optimizing!

Hello,

This article will explain how to access the Kubernetes cluster remotely using the SSH Tunnel mechanism.

The most popular tool to manage Kubernetes clusters from CLI is called kubectl. The Kubernetes command-line tool, kubectl, allows you to run commands against Kubernetes clusters. You can use kubectl to deploy applications, inspect and manage cluster resources, and view logs.

Kubernetes.io page https://kubernetes.io/docs/tasks/tools/ explains how you can install it on your desired Operating system, but today I will focus on something that I use as a Desktop, Fedora Linux.

Before you begin, please note that you must use a kubectl version that is within one minor version difference of your cluster. For example, a v1.24 client can communicate with v1.23, v1.24, and v1.25 control planes. Using the latest compatible version of kubectl helps avoid unforeseen issues.

What vendor page says https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/ you can simply use curl and download a specific version.

In my example, I've had to install the 1.15.1 version to match the Kubernetes cluster running.

curl -LO https://dl.k8s.io/release/v1.15.1/bin/linux/amd64/kubectl  

Next, we can copy kubectl to /usr/sbin or /usr/bin but we can also edit ~/.bashrc and add an alias as a workaround:

alias kubectl='/home/nemke/kubectl'  

We can test the tool, it should list basic helper commands. Next, we need to create .kube directory and copy over the config file from the Kubernetes master or worker node.

mkdir -p ~/.kube  

You can use rsync,SCP, or similar or simply copy/paste the config file and leave it under ~/.kube/config path.

Edit the config file and adjust server: section with the following:

server: https://127.0.0.1:6443  

Under - cluster: line we need to add:

insecure-skip-tls-verify: true  

And comment certificate-authority-data: if your cluster has it.

Picture of config file how it should look like:

Login to the Kubernetes master or worker node and grab the Cluster IP address from /etc/hosts file. You can use kubectl tool as well to get that info, but let's use this way.

Now it's time to setup SSH tunnel, this is working example:

ssh -f username@IP-OF-SERVER -L 6443:CLUSTER-IP:6443 -N  

When executed it will return back. That's it! Time to test...

You are now managing Kubernetes cluster from your local environment. Good luck!

Let's now test to put everything under UI. I've selected Lens because I think this IDE Kubernetes editor is brilliant when you need to troubleshoot and analyzing cluster events or simply put multiple container logs and monitor them. Lens – the Kubernetes IDE — is fruit of a Mirantis-sponsored open source project. Available for Linux, Mac, and Windows, Lens gives you a powerful interface and toolkit for managing, visualizing, and interacting with multiple Kubernetes clusters, while remaining always in proper context.

Visit https://k8slens.dev/, download your desired application, and continue.

When you log in and browse Clusters you will see your's right there listed:

Click on it and enjoy analyzing clusters and logging in to the nodes. You will still have the help of the terminal where kubectl runs and you can execute commands:

Final toughts/conclusions...

I hope this article was helpful. Using this method you can add multiple Kubernetes Clusters on your Desktop environment and manage it successfully. Good luck!

Hello my friends,

It's been a while since my last post. I wanted today to share what L2 caching in Magento is and what are two ways you can utilize it.

So what L2 caching is?
In really simple words to reduce network congestion and bandwidth usage between microservices, this mechanism is implemented using \Magento\Framework\Cache\Backend\RemoteSynchronizedCache class.

Magento stores the hashed data version in Redis, with the suffix ‘:hash’ appended to the regular key. In case of an outdated local cache, the data is transferred to the local machine with a cache adapter.

Magento or Devdocs gave really nice example on how to modify or replace the existing cache section in the app/etc/env.php file (https://devdocs.magento.com/guides/v2.4/config-guide/cache/two-level-cache.html), so basically it does what it does stores local backend to RAM, but what I want to share todate is that you can load balance between two Redis environments instead, let's say we can make one locally hosted and one on another node using well network.

'cache' => [  
        'frontend' => [
            'default' => [
                'backend' => '\\Magento\\Framework\\Cache\\Backend\\RemoteSynchronizedCache',
                'backend_options' => [
                    'remote_backend' => '\\Magento\\Framework\\Cache\\Backend\\Redis',
                    'remote_backend_options' => [
                        'persistent' => 0,
                        'server' => 'redis-somewhere-in-the-cloud',
                        'database' => '1',
                        'port' => '6379',
                        'password' => '',
                        'compress_data' => '1'
                    ],
                    'local_backend' => '\\Magento\\Framework\\Cache\\Backend\\Redis',
                    'local_backend_options' => [
                        'persistent' => 0,
                        'server' => '127.0.0.1',
                        'database' => '3',
                        'port' => '6379',
                        'password' => '',
                        'compress_data' => '1'
                    ],
                    'use_stale_cache' => false
                ],
                'frontend_options' => [
                    'write_control' => false
                ],
                'id_prefix' => 'd01_'
            ],
            'page_cache' => [
                'id_prefix' => 'd01_',
                'backend' => 'Magento\\Framework\\Cache\\Backend\\Redis',
                'backend_options' => [
                    'server' => '127.0.0.1',
                    'database' => '2',
                    'port' => '6379',
                    'password' => '',
                    'compress_data' => '0',
                    'compression_lib' => ''
                ]
            ]
        ],
        'allow_parallel_generation' => false
    ],

In my example using https://github.com/nemke82/magento2gitpod I've made a tests and indeed data is spread between two services.

Keyspace
db0:keys=1,expires=1,avgttl=11052265
db1:keys=148,expires=42,avgttl=786241446 <-- 1st Redis db3:keys=62,expires=21,avg_ttl=821121720 <-- Failover Redis environment
gitpod /workspace/magento2gitpod $ redis-cli info

Magento recommends using Redis for remote caching (\Magento\Framework\Cache\Backend\Redis) and CmCacheBackendFile for the local caching of data in shared memory, using 'localbackendoptions' => ['cachedir' => '/dev/shm/'], but as you can see we can use Redis remote or locally hosted instead and bring other benefits from it.

One of them is:
https://devdocs.magento.com/guides/v2.4/config-guide/redis/redis-pg-cache.html#redis-preload-feature

and

https://devdocs.magento.com/guides/v2.4/config-guide/cache/two-level-cache.html#stale-cache-options

Hope this article helps. Good luck optimizing!

Hello,

This is probably my last blog post in 2021. This year was really dynamic and interesting. While I was checking the Performance and SEO score for one of the clients hosted I noticed that Google is having new interesting recommendations for minifying all Javascript assets.

Terser is the JavaScript minifier. It processes JavaScript files as well as the compiled output from other languages like CoffeeScript and TypeScript and transpilers like Babel.

https://github.com/terser/terser

First, make sure you have installed the latest version of node.js (You may need to restart your computer after this step).

From NPM for use as a command-line app:

npm install terser -g  

Output:

$ node -v
v17.3.0

$ npm install terser -g

added 6 packages, and audited 7 packages in 687ms

found 0 vulnerabilities

$ whereis terser
terser: /usr/local/nvm/versions/node/v17.3.0/bin/terser  

Usage is really simple:
https://github.com/terser/terser#command-line-usage

Terser can take multiple input files. It's recommended that you pass the input files first, then pass the options. Terser will parse input files in sequence and apply any compression options. The files are parsed in the same global scope, that is, a reference from a file to some variable/function declared in another file will be matched properly.

If no input file is specified, Terser will read from STDIN.

If you wish to pass your options before the input files, separate the two with a double dash to prevent input files from being used as option arguments:

terser --compress --mangle -- input.js  

In Magento 2 specifically, we can implement the following after setup:static-content:deploy (Static asset) deployment is processed, but before that, we need to know the path of our Theme directory.

For example, let's try now to bundle the default Luma theme on a specific path for en_US language:

THEME_FOLDER=('/workspace/magento2gitpod/pub/static/frontend/Magento/luma/en_US')  

Now, let's bundle files with the following line:

find ${THEME_FOLDER[@]} \( -name '*.js' -not -name '*.min.js' -not -name 'requirejs-bundle-config.js' \) -exec /usr/local/nvm/versions/node/v17.3.0/bin/terser \{} -c -m reserved=['$','jQuery','define','require','exports'] -o \{} \;  

Note: we skipped some of the patterns like *.min.js and requirejs-bundle-config.js, but if you are using MagePack or any other popular Bundling tool try to skip their Core bundle files generated. For example, for Baler tool:

-not -name 'core-bundle.js'

You can test and Minify them as well, but my advice is to test this before adding this to your Production environment.

The last step is to clear Magento 2 cache and Redis if used and test.

Run audit test again, let's say on https://web.dev/measure/ website and compare results before/after

Disable built-in Minify mechanism for Javascript minify and integrate this tool and how to convert your Theme into your Deployment process and enjoy the performance boost.

\o/ Hope this was helpful.