Yes, but I am looking for one way of doing it, which I suspect is the minimum number of components.
This is what I had in mind. Analogously to the 2D case, join the innermost vertices of the cut out cubes to form a cube in the center. It will have length a – 2b. Let’s call this cube C.
Consider the top face of C. It has two sides of length a – 2b. Extend this face upward until it meets the original cube. This gives a rectangular solid with lengths a-2b, a – 2b and b. Repeat this for each of the remaining five faces of C. We end up with 6 rectangular solids with lengths a-2b, a-2b and b.
Now comes the difficult part of the visualization. Consider the edge between the top and left faces of C. We now have a region running along this edge between the solids we just constructed on the tops of these two faces. It is a rectangular solid of lengths a-2b, b and b. There are 12 of these solids, one for each edge of C, and we are now done.
One way of checking that our visualization is correct is to see if the sum of the volumes of the components, including the 8 cubes we originally removed, add up to the volume a cubed. This looks like a lot of work, but there is a way of simplifying the work. We note that a = (a-2b)+2b, so a cubed is the cube of the expression on the right. If you cube the right hand side, you get 4 expressions in terms of powers of (a-2b) and b. These coincide with the volumes of the 4 types of components of a, and the coefficients of 1, 6, 12 and 8 match the number of each type of component we used.