We propose to use convolutional neural networks to detect contaminants in astronomical images. Once trained, our networks are able to detect various contaminants such as cosmic rays, hot and bad pixels, persistence effects, satellite or plane trails, residual fringe patterns, nebulosity, saturated pixels, diffraction spikes and tracking errors in images, encompassing a broad range of ambient conditions (seeing), PSF sampling, detectors, optics and stellar density. MaxiMask is performing semantic segmentation: it can output a probability map for each contaminant, assigning to each pixel its probability to belong to the given contaminant class, except for tracking errors where another convolutional neural network can assign the probability that the entire focal plane is affected. Training and testing data have been gathered from real data originating from various modern CCD and near-infrared cameras or simulated data. We show that MaxiMask achieves good performance on test data and propose a prior modification technique based on Bayesian statistics to optimize its behaviour to any expected class proportion in real data.