MySQL 8 Network Backup Using Docker and CLONE

One of the shortcomings of MySQL GPL is that it does not come with a first party online backup solution. With the release of MySQL 8.0.17 the CLONE plugin was introduced, this essentially integrated online backup as a plugin to the MySQL Server.

The MySQL 8.0 Reference Manual describes how to use the CLONE plugin to perform local clones (backups) here:

Doing local clones is incredibly useful and a really fast way of making an image backup. I would argue that the CLONE plugin is better for local image backups than competing solutions simply because the syntax is more brief and efforts were made to integrate CLONE into the server, thereby reducing the impact of performing CLONE operations.

The CLONE plugin can either clone to the server’s default data directory or to another directory specified in the CLONE command. I will demonstrate the latter usage for making online remote backups without modifying the data directory of the container.

Continue reading “MySQL 8 Network Backup Using Docker and CLONE”

Bona Fides: Linux Kernel

This page shouldn’t be considered a brag page, it’s just a place for me to easily categorize a Linux Kernel contribution I made eons ago. This is my original contribution of the vfork(2) system call. The current Linux kernel does not implement it in this way, however syscall 190 is still sys_vfork ?

Subject: [PATCH] new syscall: sys_vfork
To: (Linux Kernel Mailing List)
Date: Fri, 8 Jan 1999 10:49:54 -0800 (PST)
X-Mailer: ELM [version 2.4 PL24]
Content-Type: text
Status: RO
Content-Length: 5783
Lines: 156


Well, I hacked in support for a traditional style vfork.  I haven't
tried actually running an application using the new vfork; I wanted
to release what I have to get feedback, as this is the first patch
I've really done.

Anyhow, some background first:

This implementation of vfork supports these features:

 - the VM is cloned off the parent
 - the parent sleeps while the vfork()ed child is running
 - the parent awakes on an exec() and exit()
 - the implementation theoretically allows for recursive vforks
 - it's executable from within a cloned thread
 - If I'm right about the flags, the sigmask is not cloned

A little bit about the 'controversial' parts:  The implementation
uses a wait queue in the task structure.  When the parent vforks,
after successful spawning, it sleeps on the vfork wait queue.  When
the child exits or execs, it does a wake_up(&current->p_pptr->vfork_sleep);
Which causes the parent to awake.  The wakeup in the exec is right
at the top of do_execve().  The wakeup in exit is right before
the time the parent gets notified of the child exit (before notify_parent);

It allows recursion because if a vforked child vforks, it just sleeps,
and as each vforked child performs an exec or exit, it percolates up
through the vfork execution stack.

Please let me know if I've done anything grossly wrong, or just wrong.
Additionally, could someone tell me how to do direct syscalls, I'm fuzzy
on that ;)



diff -u --recursive linux.vanilla/arch/i386/kernel/entry.S linux/arch/i386/kernel/entry.S
--- linux.vanilla/arch/i386/kernel/entry.S      Thu Jan  7 19:21:54 1999
+++ linux/arch/i386/kernel/entry.S      Thu Jan  7 20:38:18 1999
@@ -559,13 +559,14 @@
        .long SYMBOL_NAME(sys_sendfile)
        .long SYMBOL_NAME(sys_ni_syscall)               /* streams1 */
        .long SYMBOL_NAME(sys_ni_syscall)               /* streams2 */
+       .long SYMBOL_NAME(sys_vfork)            /* 190 */

-        * NOTE!! This doesn' thave to be exact - we just have
+        * NOTE!! This doesn't have to be exact - we just have
         * to make sure we have _enough_ of the "sys_ni_syscall"
         * entries. Don't panic if you notice that this hasn't
         * been shrunk every time we add a new system call.
-       .rept NR_syscalls-189
+       .rept NR_syscalls-190
                .long SYMBOL_NAME(sys_ni_syscall)
diff -u --recursive linux.vanilla/arch/i386/kernel/process.c linux/arch/i386/kernel/process.c
--- linux.vanilla/arch/i386/kernel/process.c    Thu Jan  7 19:21:54 1999
+++ linux/arch/i386/kernel/process.c    Thu Jan  7 20:33:23 1999
@@ -781,6 +781,19 @@
        return do_fork(clone_flags, newsp, &regs);

+asmlinkage int sys_vfork(struct pt_regs regs)
+       int     child;
+       child = do_fork(CLONE_VM | SIGCHLD, regs.esp, &regs);
+       if (child > 0) {
+               sleep_on(&current->vfork_sleep);
+       }
+       return child;
  * sys_execve() executes a new program.
diff -u --recursive linux.vanilla/fs/exec.c linux/fs/exec.c
--- linux.vanilla/fs/exec.c     Sun Nov 15 09:52:27 1998
+++ linux/fs/exec.c     Fri Jan  8 10:32:59 1999
@@ -808,6 +808,9 @@
        int retval;
        int i;

+       /* vfork semantics say wakeup on exec or exit */
+       wake_up(&current->p_pptr->vfork_sleep);
        bprm.p = PAGE_SIZE*MAX_ARG_PAGES-sizeof(void *);
        for (i=0 ; i<MAX_ARG_PAGES ; i++)       /* clear page-table */
      [i] = 0;
diff -u --recursive linux.vanilla/include/linux/sched.h linux/include/linux/sched.h
--- linux.vanilla/include/linux/sched.h Thu Jan  7 19:27:44 1999
+++ linux/include/linux/sched.h Thu Jan  7 21:57:20 1999
@@ -258,6 +258,10 @@
        struct task_struct **tarray_ptr;

        struct wait_queue *wait_chldexit;       /* for wait4() */
+/* sleep in vfork parent */
+       struct wait_queue *vfork_sleep;
        unsigned long policy, rt_priority;
        unsigned long it_real_value, it_prof_value, it_virt_value;
        unsigned long it_real_incr, it_prof_incr, it_virt_incr;
@@ -298,6 +302,7 @@
        struct files_struct *files;
 /* memory management info */
        struct mm_struct *mm;
 /* signal handlers */
        spinlock_t sigmask_lock;        /* Protects signal and blocked */
        struct signal_struct *sig;
@@ -349,6 +354,7 @@
 /* pidhash */  NULL, NULL, \
 /* tarray */   &task[0], \
 /* chld wait */        NULL, \
+/* vfork sleep */      NULL, \
 /* timeout */  SCHED_OTHER,0,0,0,0,0,0,0, \
 /* timer */    { NULL, NULL, 0, 0, it_real_fn }, \
 /* utime */    {0,0,0,0},0, \
diff -u --recursive linux.vanilla/kernel/exit.c linux/kernel/exit.c
--- linux.vanilla/kernel/exit.c Tue Nov 24 09:57:10 1998
+++ linux/kernel/exit.c Fri Jan  8 10:34:10 1999
@@ -292,6 +292,10 @@
+       /* notify parent sleeping on vfork() */
+       wake_up(&current->p_pptr->vfork_sleep);
        /* Let father know we died */
        notify_parent(current, current->exit_signal);

diff -u --recursive linux.vanilla/kernel/fork.c linux/kernel/fork.c
--- linux.vanilla/kernel/fork.c Thu Jan  7 19:27:29 1999
+++ linux/kernel/fork.c Thu Jan  7 20:24:53 1999
@@ -521,6 +521,7 @@
        p->p_pptr = p->p_opptr = current;
        p->p_cptr = NULL;
+       init_waitqueue(&p->vfork_sleep);

        p->sigpending = 0;


Reducing the Impact of YouTube’s API Quota

I started redesigning my website a several weeks ago, my objective was to create a centralized hub for sharing written information, code, video, and photography. It was rather easy to solve most of those problems, and sharing my latest YouTube video was simple at first.

I had this niggling feeling that my new website was on the heavyweight side, after all it’s WordPress based and I had a few plugins. The annoying reCaptcha logo was popping up everywhere, even when it wasn’t used. After using the Coverage tab in Chrome and installing yet more WordPress plugins to trim the fat, I tried get it down to as small a footprint as I could. Then came the Google PageSpeed Insights. Sometimes we are blissfully unaware of our problems and go through life with blinders on, PageSpeed Insights simultaneously woke me and gave me yet another obsession to chase.

Continue reading “Reducing the Impact of YouTube’s API Quota”

Adding VGA hardware palette support

VGALIB has lead a long and meandering path, development has been an exercise of leveling up each of 3 different environments: PC hardware running DOS, SDL under Linux, and SDL under emscripten. Much of the early development was done in dosbox with the Borland C++ 3.1 IDE, but once I grew past the point of basic C++, using std::string, I had to abandon the BC3.1 IDE and go strictly to makefiles. It was during this time that using the BC3.1 IDE for editing (and it’s weird Brief key sequences) started to become an exercise in patience. I really enjoyed developing on Linux, since that’s what I’ve done for the last 25 years.

Moving to makefiles under DOS was no small feat, the issue is that dosbox is a best effort emulator for running games, but compatibility with Borland C++ 4 and later is sketchy causes crashes. I ended up creating a Windows 2000 VM with Virtualbox to compile VGALIB, but even that acts peculiar and cmd.exe requires End Task. Virtualbox doesn’t have guest additions for any 16bit legacy OSes, so Win2K is the oldest usable environment. My current development environment is Eclipse for the editing (with VIM plugin), Win2K to compile the DOS programs, and dosbox to run them. For Linux and emscripten I use Eclipse with command line make.

The reason my build environment is important to this article has to do with the development target that was most feature complete: SDL running on Linux. Palettized 8bit mode on SDL is really a pain to program to, much more so than straight RGB or RGBA, but it mimics the original IBM VGA 13h mode most closely. I implemented palette support as a matter of requirement when I added SDL support, since there there is no default palette. Until this time I hadn’t added hardware palette support to the VGA driver, I simply relied on the default VGA palette (which is fine for most things).

Continue reading “Adding VGA hardware palette support”

The Sale of WebCom

The sale of WebCom was both bitter and sweet. The sale represented independence and success for many involved, but it also was the beginning of the end. WebCom was bootstrapped from what money Chris had and some surplus equipment that we got from a customer in exchange for free hosting. That equipment lasted us until late 1995 when we needed to transition from a 486 running Windows NT 3.51 and Microsoft SQL Server, to a Sun Enterprise 1000e running Sybase SQL Server.

I mentioned before that Chris and Thomas organized the company with a 67%/33% split, eventually I would have 1%, taken from Chris’ portion, and Neal [the CFO] got 10% IIRC, of which I think Chris and Thomas gave up 5% each. After we moved to 2880 Soquel Ave, Thomas started working on his exit from the company. That exit would precipitate one of the biggest threats we ever had as a company.

Continue reading “The Sale of WebCom”

Hacking CGA

This is meant to be a short post to talk about some CGA idiosyncrasies and how you can bypass them.

My video library VGALIB now supports CGA in addition to VGA, EGA support is planned too. Adding VGA was simple and that’s why I did it first; VGA implements a 320×200 linear framebuffer. A linear framebuffer is one where each pixel is represented by a simple lookup and the pixels are contiguous in the memory region. The formula width*y + x is commonly used to perform linear buffer address resolution. It is because of this simplicity that I made the internal representation of images 8 bit linear buffers. Each pixel is represented by 1 byte that can hold 1 of 256 colors.

Continue reading “Hacking CGA”

WebCom: New Locations, New Logos, Questionable OpSec

The only constant in a startup is growth, and WebCom grew exponentially during the 4 years it was WebCom. Recapping a little bit, when I joined WebCom it was just Chris and Thomas, shortly after came Rick. There were 3 people in the A suite and Rick was in the B suite, more employees wouldn’t fit! In late 1995 we moved from 903 Pacific to 125 Water St, the new location was a lot bigger but shared the same pains: parking. Working in downtown Santa Cruz is an exercise in patience, strategy, and luck. Sometimes all 3 are on your side and sometimes all 3 are against you!

The new location was a welcomed change from the office full of hand-me-down furniture, we got cubicles! Yes, it seems strange to be excited by cubicles, but it meant that Rick and I got nearly double the space we had before. The A suite had 2 offices, one that was a nice professional office and one that had the telephone lines and other miscellany — the latter became the break room and server room, with a cubicle partition in the middle. Thomas was the proud tenant of an 8×12 cubicle office that didn’t quite reach the ceiling. I think we had about 13 cubicle desks at that location, which lasted us until late 1997, when we were bursting at the seams.

Continue reading “WebCom: New Locations, New Logos, Questionable OpSec”

Web Communications

Web Communications [WebCom] was a web hosting provider that started in Santa Cruz California in late 1994, opening to the public in 1995. WebCom was the brainchild of Chris Schefler, a Cal State graduate who believed in freedom, communication, and ecology.

Chris started WebCom with co-founder Thomas Leavitt in a small windowless office at 903 Pacific Ave, Suite 306 A. This building was informally named Geek Hall because it was the nexus of every internet connected Santa Cruz startup.

Continue reading “Web Communications”

Introduction to VGALIB

During the Christmas break of 2018 I started working on a cross-platform video library that would work on DOS, Linux, and Web. The idea was really born from tinkering with some old C++ code that found on an old hard drive.

I wrote my first C++ code in 1995 in Borland C++ 3.1 on my old DOS computer. When I was a Junior in High School [ca 1993] I was part of what they call a “coding bootcamp” today. There were 2 Support Engineers from Borland, Tom Orsi and Jeff Peters, and they volunteered to teach a small group of HS students how to program in C. Most, but not all of us, had prior programming experience in another language. My experience was with Microsoft QBasic, Visual Basic for DOS, and Visual Basic 3.0 for Windows.

Continue reading “Introduction to VGALIB”

Fadal 1400-1 CPU Card Reverse Engineering and Schematic

This article originally appeared in My little (home) shop thread on Practical Machinist.

I only worked on the CPU board schematic a few minutes today, but I finished all the address decoding logic.

The 8088 only has 20 address lines, of that, 16 of them are used for addressing segments of 64KB each. The address lines A16 through A19 are used as card selects to talk to the peripheral cards. There is a 4 to 16 line decoder that takes the upper nibble of the CPU address, then converts that to a selector that is 1 of 16 lines on the bus. Fadal repurposed a bunch of address and logic lines to make these card selects.

A 20bit address is represented as 5 hex nibbles, like 0xFBC00, that’s segment 15, address BC00. That happens to correspond to the video memory on the video card. The first F selects card 15, which is the video card (we are talking base 0 numbers here, so 0-15), the BC00 selects the base address within the F segment. The F segment also contains the system ROM, which is located at FE000-FFFFF, an 8KB chunk. The video ram is only 1KB, so it occupies addresses FBC00-FBFFF.

It would seem that the memory expansion cards occupy a card select range, though I’m not certain exactly where the memory and NC executive ROMs are mapped yet. Once I have the schematic completed and the system ROM reverse engineered, I could design my own memory expansion card.

Continue reading “Fadal 1400-1 CPU Card Reverse Engineering and Schematic”