Dyalog APL, 24 bytes
Takes matrices A and B as the left and right args respectively
{↑(↓⍺){(⊂⍺)+/⍤ר⍵}¨⊂↓⍉⍵}
↓⍉⍵ # Transpose the right arg and split into vector of vectors
(↓⍺) # Split the left into vector of vectors as well
{ }¨⊂ # Combine each left with all of the right
(⊂⍺) ¨⍵ # For each of the right and all of the left
+/ # Sum
⍤ # after
× # multiplying
↑ # Mix back into a matrix
💎
Created with the help of Luminespire.