Could it be just a coincidence that the exclamation mark is the earliest non-system and non-whitespace character in the ASCII table (at #32)? Would this characteristic make parsing quicker?
No, most language design committees would prioritize an easy-to-learn, remember, and type syntax over any slight advantage gained from smaller byte values.
Some hand-rolled parsers may utilize lookup-tables to classify tokens based on the first character. For instance, Mozilla's JS engine includes definitions in jsscan.cpp for the lexer:
static const uint8 firstCharKinds[] = {
/* 0 1 2 3 4 5 6 7 8 9 */
/* 0+ */ _______, _______, _______, _______, _______, _______, _______, _______, _______, Space,
/* 10+ */ EOL, Space, Space, EOL, _______, _______, _______, _______, _______, _______,
/* 20+ */ _______, _______, _______, _______, _______, _______, _______, _______, _______, _______,
/* 30+ */ _______, _______, Space, _______, String, _______, Ident, _______, _______, String,
/* 40+ */ OneChar, OneChar, _______, Plus, OneChar, _______, Dot, _______, HexOct, Dec,
/* 50+ */ Dec, Dec, Dec, Dec, Dec, Dec, Dec, Dec, Colon, OneChar,
...
However, opting for | instead of ! will not significantly reduce the size of such tables. CSS and HTML are resource-intensive, so devices with memory constraints tend to avoid parsing CSS. Therefore, saving a few bytes ('|' - 'z' == 2) in the lexer is unlikely to have a substantial impact on browser performance.