When St. James (a.k.a. St. Jacques, or Santiago) went wandering on religious pilgrimages, he took with him the shell of a sea scallop — the cuplike bottom half. If he asked for food or drink to sustain him along the way, he would only accept the small amount that fit in the shell. The scallop shell has been the symbol for him ever since.
On the origin of the French name for scallops: Coquilles St. Jacques
Courtesy of the NY Times