Preheat the broiler and move your oven rack to the top of the oven. In a cast iron skillet or on a sheet pan, toss the peppers, olive oil and salt. Broil for 4 minutes, then turn the peppers with tongs. Broil for another 4 minutes. Toss the peppers with a little more salt.

For the dipping sauce, stir together the mayonnaise, sriracha and lime juice. Serve the peppers with the sauce alongside.