When examining the submit button element in Firebug, it appears that the elements touch without any margin. Despite this, there is a 3px wide border present. If you closely inspect the border by taking a screen capture and zooming in, you will notice a 1px wide gray section with transparent parts on either side, excluding at the rounded corners.
Browsers typically use built-in routines to draw default borders, resulting in a specific appearance. However, if you modify the button's border properties using CSS, the browser may switch to a standard border drawing method, altering the button's appearance accordingly. While it may be difficult to entirely replicate the default border look, adjusting the following CSS properties can get you fairly close (on browsers that support it):
input[type="submit"] {
border: solid gray 1px;
border-radius: 4px;
background: #ddd;
}
Check out the demo here: http://jsfiddle.net/m2CSy/