Pros and cons of SVG for images

When used as a document format there is usually a compelling reason that makes SVG the only solution. When used as an image format, it is sometimes less obvious whether it would be best to use SVG or a raster image format for any given image. The vector format SVG and raster formats like PNG both have their place. When choosing whether or not to use SVG it is best to understand the advantages and disadvantages of both.

File size
Whether SVG or a raster format will produce a smaller file for a given image depends very much on the image. For example, consider an image of a path with a gradient fill. The size of an SVG of this image will be the same regardless of the dimensions of the image. On the other hand the size of a raster file of the same image will likely vary tremendously depending on the dimensions of the image since the larger the dimensions the more pixel data the file needs to store. At very small dimensions (the extreme case being 1px x 1px) the raster file will likely be much smaller than the SVG file since it only needs to store one pixel of data. At large dimensions the raster file may be much larger than the SVG file.
Scalability, with caveats
One of the primary advantages of SVG is that as it is scaled it does not pixelate. However, this is not to say that it always does away with the need to have a collection of raster images for display at different scales. This can be particularly true for icons. While SVG may scale well enough for flat-ish icons without a lot of detail, for icons that try to pack in a lot of detail graphic artists generally want to be able to pixel tweak.
Performance
While SVG provides a lot of flexibility in terms of scaling, themability, etc. this flexibility depends on doing computations for SVG images at the time they're displayed, rather than at the time the author creates them. Consider an image that involves some complex gradients and filters. If saved as a raster image then the work to rasterize the gradients and filters takes place on the authors computer before the result is stored in the raster file. This work doesn't need to be redone when the image is displayed on someone else's computer. On the other hand, if the image is saved as an SVG image then all this work needs to be done each time the SVG is displayed on someone else's computer. This isn't to say that SVG images are always slower than raster equivalents. In fact it can be faster to send vector information from an SVG to a user's GPU than it is to extract raster data from an equivalent raster image. And even when an SVG image is slower than a raster equivalent, the difference is usually not noticeable. However, just don't fall into the trap of thinking that SVGs are faster than equivalent raster images, or vice versa. Once again, "it depends".

Authoring guidelines

A lot of SVG files (particularly those generated by SVG editors) ship without being cleaned up and can contain a ton of junk that bloats the file size and slows down rendering. In general the best way to combat this is to first run SVG files through a linter such as svgo (see the Tools section below). However, when authoring SVGs by hand here are some best practices to help keep them lightweight. These rules are based on some real examples seen in Mozilla's code.

Basics

Whitespace and line breaks

Whitespace

In addition to trailing whitespace at the end of lines, there are a few more cases more specific to SVGs:

Examples

This path: 

<path d=" M5,5    L1,1z ">

can be cut down to this:

<path d="M5,5 L1,1z">

Similarly, this polygon:

<polygon points="  0,0   4,4  4,0  "/>

can be cut down to this:

<polygon points="0,0 4,4 4,0"/>

Line breaks

You should only use line breaks for logical separation or if they help make the file readable. You should avoid line breaks between every single element or within attribute values. It's recommended to put the attributes on the same line as their tag names, if possible. You should also put the shortest attributes first, so they are easier to spot.

Unused tags and attributes

Editor metadata

Vector editors (Inkscape, Adobe Illustrator, Sketch) usually add a bunch of metadata in SVG files while saving them. Metadata can mean many things, including:

Other metadata

In addition to non-standard editor metadata, standard compliant metadata also exists. Typical examples of this are <title> and <desc> tags. Although this kind of data is supported by the browser, it can only be displayed when the SVG is opened in a new tab. Plus, in most of the cases, the filename is quite descriptive So it's recommended to remove that kind of metadata since it doesn't bring much value.

You shouldn't include DOCTYPEs in your SVGs either; they are a source of many issues, and the SVG WG recommends not to include them. See SVG Authoring guidelines.

Avoid the use of CDATA sections

CDATA sections are used to avoid parsing some text as HTML. Most of time, CDATA isn't needed, for example, the content in <style> tags doesn't need to be wrapped in a CDATA section as the content inside the tag is already correctly parsed as CSS.

Invisible shapes

There are two kinds of invisible shapes: The off-screen ones and the uncolored ones.

The offscreen shapes are hard to spot, even with an automated tool, and are usually context aware. Those kinds of shapes are visible but off the SVG view box. Here's an example of a file with offscreen shapes.

On the other hand, the uncolored ones are easier to spot, since they usually come with styles making them invisible. They must meet two conditions: they must be devoid of any fill (or a transparent one) or stroke.

Unused attributes on root <svg> element

The root <svg> element can also host many useless attributes. Here's an example taking into account the list below:

Other

Styling

Basics

Examples

Here are some examples for excessive number precision:

As for descriptive IDs:

Use of class names

Default style values

There's usually no need to set the default style value unless you're overriding a style. Here are some commonly seen examples:

SVG grouping

Style grouping

Group similarly styled shapes under one <g> tag; this avoids having to set the same class/styles on many shapes.

Avoid excessive grouping

Editors can sometimes do excessive grouping while exporting SVGs. This is due to the way editors work.

Nested groups

Avoid multiple-level nesting of groups, these make the SVG less readable.

Nested transforms

Some editors use <g> tags to do nested transforms, which is usually not needed. You can avoid this by doing basic algebra, for example:

<g transform="translate(-62, -310)"><shape transform="translate(60, 308)"/></g>

can be cut down to:

<shape transform="translate(-2,-2)"/>

because: -62+60 = -310+308 = -2

Performance tips

These rules are optional, but they help speeding up the SVG.

Tools

Tools can help to clean SVG files. Note, however that some of the rules stated above can be hard to detect with automated tools since they require too much context-awareness.
To this date, there doesn't seem to be a tool that handles all of the above. However, there are some utilities that cover parts of this document: