This is a very interesting question. There is, indeed, an issue with establishing the security, veracity and stability of Firefox and other Mozilla product extensions, now known as add-ons. For example, in July of 2006 a fake Firefox add-on was discovered to have attempted to steal passwords and perform key logging after installation. This add-on was particularly worrying because it was capable of being installed without prompting the user.
Unfortunately, there isn't a ready way to determine whether an extension is malicious. One of the few ways is doing basic research. I recommend using Google or your choice of search engine to find information about the add-on you wish to use. If the add-on is malicious, hopefully someone else has encountered it before and has identified its malicious nature. Checking the sites for the major anti-virus vendors is also recommended. They are often the first groups to identify threats and alert on them.
Probably the (comparatively) safest way to download and install add-ons is via the Mozilla Add-Ons site. Add-ons to the site should have a sponsor who tries to ensure the add-on is of a sufficient quality and appropriate nature. A review process should take place before plug-ins are added and available for download. However, the results of this review process do not appear to be documented anywhere, nor does there appear to be a standard security policy for Mozilla add-ons.
It should also be noted that, while Mozilla has a policy for the management and review of add-ons, it does not offer any liability in the event something goes wrong -- either in terms of functionality or security. The policy is also in draft.
The other obvious mitigation for potentially malicious add-ons is the use of good anti-virus and personal security tools, like personal firewalls, anti-spam and anti-spyware tools. The latter are strongly recommended. Many of these tools will alert you to malicious activity or prompt you for a response if unusual activity is detected. You then have the option of denying that activity.
In summary, the use of most Firefox (and Thunderbird and other Mozilla tools) add-ons is at your own risk. This is a great shame as they represent some excellent and useful enhancements to the Mozilla suite of products. If, like me, you are a fan of and are serious about making use of Mozilla products and the add-ons for them, then I recommend you contact Mozilla and suggest that they put in place a transparent and formalized process for the submission, review and certification of add-ons. This should be backed up by digital signatures for add-ons that have passed this review and been "certified" for use with Mozilla products. I would suggest the addition of this process would also likely increase the level of quality control for Mozilla add-ons.
This was first published in November 2006