Setting a custom color for the git gutter fringe creates double fringe on doom/reload

I tried to set a custom color for the git gutter like this:

(custom-set-faces 
  '(diff-hl-insert ((t (:foreground "#00ff00"))))
  '(diff-hl-change ((t (:foreground "#ffff00"))))
  '(diff-hl-delete ((t (:foreground "#ff0000"))))
)

This works when Emacs is started, but after that if I run doom/reload after making more changes, I get a double fringe (one with my color, and one with the default color). I’m not sure what’s going on here.

Isn’t this overriding the default settings the correct way?