Idk the colors look fine to me, but if you want to reduce the contrast a little maybe choose a slightly lighter shade of brown to bring the whole palette to a similar level?

Really if you want colors to work together it's all a matter of having them compliment each other, the color wheel can really help there. Most people only thing of the bright primary colors but the more saturated shades like the greys and browns can be mapped onto it too just like any bright primary color

https://lh5.googleusercontent.com/_rowX57E4gy0/TYazAANmpoI/AAAAAAAAAlU/1W0G-MIA6Gg/s512/FullColorWheel.png
http://bellavieboutique.files.wordpress.com/2012/07/colorwheel.jpg

The colros most people associate with brown are more towards the red end of the spectrum, but using the other major color as pink here the color brown that'd naturally compliment it would more be closer to the purples inbetween red and blue.

Also triads and other relations help work too, general relations between colors - imagine you drew a big triangle with all the sames the same length on the color wheel with each point on a different color
http://www.tigercolor.com/Images/Triad.gif

These create color harmonies and tend to work well together. You could use this with your pink color and try to find where a good contrasting saturated color might lie on the color wheel on the outside that'd still compliment the brighter pink.

At least that's some of the stuff I'm remembering from painting, it's worth a shot but idk how useful it'll be

that's enough bows

cheat with http://paletton.com/ =w=

also a good guide here
http://stuff.veekun.com/pkcolor/