Notes from the Field

SMTP 5.1.1. The e-mail account does not exist at the organization this message was sent to.

2008-01-18T15:53:00Z

This was a nice little problem, and a neat fix, so I thought I'd share it.

I got a call from a customer who was having a small problem with a senior manager, which always seems to be the way!

The managers PA had left, and the user object and Mailbox had been deleted and purged.

Now, everytime anybody booked a meeting/sent a meeting request to Mr Manager, the sender got an NDA from the missing PA!

The error was " The e-mail account does not exist at the organization this message was sent to. " which makes sense, with an SMTP status code 5.1.1

5.1.1 is a fairly new code, I won't break them all out here, but 5.x.y is a fatal error, and the first 1 is an information report. 5.1.1 basically is informing you that the mail didn't get thru, because the user doesn't exist, which is no real surpirse.

The reason this happens is pretty simple.

When you set up delegate access to a calendar using the Delegate Access Wizard in Outlook, you have options (depending on which version of Outlook) to have meeting requests sent to your delegate.

The wizard sets up a hidden rule within the mailbox.

If the delegate is deleted before the delegate permissions are removed, then the rule is orphaned.

There is a download detailed in PSS ID: 253557 http://support.microsoft.com/kb/253557/en-us that walks you thru the MDBView utility, which can be fun, but try this first.

Click the Rules button.

There is a rule listed with <no name>

Delete it.

Problem solved!

LDAP OR what?

2008-01-17T14:28:00Z

Using LDAP OR in Address lists.

This has bugged me for a little while.

Imagine the following scenario...

You have several departments, or entities, or even companies hosted in a single Exchange 2003 organisation.

You want to provide each with their own seperate address list. Out of the box you get four

All contacts
All groups
All users
Public folders

By default, the LDAP Query options in ESM use a logical AND, and this can be restrictive.

Let me give you my real world example.

I have a seperate mailbox store database SG1DB2 just for the members of a specific department in a university, where the GAL contains over 60 thousand student mailboxes.

This dept is happy with the GAL, we don't have to do anything clever with permissions or address list views, but they have requested that in addition to the four address lists displayed above, that I create a fifth departmental list.

So I right click All Address Lists in ESM, and select New, Address List... and give it an Address List name: and select Filter Rules...

On the General tab I tick the box for Users with Exchange mailbox and on the Storage tab, under Mailboxes in this mailbox store: I select SG1DB2.

OK, Finish and we're done, everybody is happy, especially me, 'cos it was easy.

Then, the phone rings, and now this department thinks it would be kinda nice if the department address list also contains the department distribution groups.

So I open ESM, right click my newly created address list and get Properties.

I select Modify... and go to the Advanced tab

I drop down Field to Group and in my case select Name, under Condition: I select starts with, and in Value: I type ABC, because in my case all of the departmental groups in question start with ABC.

I select Add and OK, and then to check my work I select Preview... and guess what, empty address list. It's blank!

What happens here is that I'm now looking for all of the users in SG1DB2 that are also a group starting with ABC! ESM has used a logical AND filter in the LDAP query, and of course the results are useless.

So now I have to figure out the LDAP filter to give me all mailboxes in a particular mailbox store, OR any department starting with ABC.

The LDAP for AND is &. For OR it's |

The simplest way to do this, and I appreciate it depends on the complexity of the LDAP query you are trying to perform, is to do the following.

Do two seperate queries.

The first one is for mailboxes in a particular mailbox store, which is where we started. On the Properties for the address list, you can select the LDAP query and copy it to notepad.

(&(&(& (mailnickname=*) (| (&(objectCategory=person)(objectClass=user)(homeMDB=CN=SG1DB2,CN=First Storage Group,CN=InformationStore,CN=EXCH01,CN=Servers,CN=First Administrative Group,CN=Administrative Groups,CN=University,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=UNI,DC=LOCAL)) ))))

Next, I Remove the query, and click Modify... and from the Find: drop-down list select Custom Search

Now I re-enter my query for Group Name starts with ABC, press Add and OK

Now select this new LDAP query and copy it into the same notepad, on the next line down.

(&(objectCategory=group)(cn=ABC*))

Now, nearly there. We know that LDAP for OR is | so we need to stick these two conditions together.

(| (Condition one)(Condition two))

Open and close brackets with an OR statement, that's it, and it's far easier than trying to figure out how to do it from within the options available in ESM.

One final step. Now that we have our LDAP query:

(| (&(&(& (mailnickname=*) (| (&(objectCategory=person)(objectClass=user)(homeMDB=CN=SG1DB2,CN=First Storage Group,CN=InformationStore,CN=EXCH01,CN=Servers,CN=First Administrative Group,CN=Administrative Groups,CN=University,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=UNI,DC=LOCAL)) ))))(&(objectCategory=group)(cn=ABC*)))

we need to get ESM to use it, and that's not so simple.

In theory it's a question of creating a Custom Search and simply pasting the new query into the advanced tab, but I get some funnies when I try this, it either doesn't accept the paste, or it does but it modifies it to include an additional (& at the start, neither of which work for me, the list always previews as empty!

So, open ADSIEDIT, and browse the Configuration container.

CN=Services, CN=Microsoft Exchange, CN=Organisation Name, CN=Address Lists Container, CN=All Address Lists.

Right click the address list in question and get Properties

Find the purportedSearch attribute, and Edit

Press Clear and paste the new LDAP in here. OK twice, close ADSIEDIT and we're done.

Now when you preview the Address List in ESM, it should contain mailboxes in a particular store, OR groups that start with ABC!

Simple ;-)

Making sense of Mailbox Access in Exchange 2003

2007-10-10T09:50:00Z

Delegates, shared folders, and sending mail as somebody else.

I’m currently working on an Exchange 2003 deployment that has 1800 staff and 50,000 student mailboxes.

We have had so many generic requests for what has come to be known simply as “Delegate Access” that I’ve put a guide together for the project, and hopefully it will be of some use to other people so I thought I’d share it.

Let me start by taking a step backwards.

Outlook 2003 and Exchange 2003 are only 30% of the solution.

People, Process, Technology. I know it’s a cliché, but it’s very true. Out of the box Outlook and Exchange can do some very clever things, but unless you put a bit of thought into what you are trying to achieve, then no amount of ticked boxes or level of granular permission is guaranteed to work for you.

The Internet is full of custom guides on how to set up delegate access to mailboxes, and various ways to open other user’s folders, or open additional mailboxes.

What we have struggled to find for this project is less technical and more business oriented guidance on when each of the various options is best used, and what the main differences between them are.

In the Beginning

Everything in a Microsoft world depends on principle based security. A security principle is an Active Directory object with a security identifier or SiD. Microsoft recognises three security principles, the user, the computer and the security enabled group. These are the only three Active Directory objects that have SiD’s, and these are the only objects that can be assigned security tokens. A security token is used to allow or restrict access to resources based on Access Control Entries (ACE’s) stored in Access Control Lists (ACL’s)

Way back in 1993 Microsoft decided that a username and password uniquely identified an individual, and today with Active Directory a mailbox is simply an extension of a user object. It’s owned by the uniquely identified individual.

I digress a little bit here but it becomes relevant in the whole delegate access mailbox conversation because many requests for delegate access evolve from an attempt to implement role based security. For example, on Monday the helpdesk support function is performed by Joe, on Tuesday it’s performed by Fred, on Wednesdays it’s either Joe or Fred, and John actually does the role all week.

So we have multiple people, multiple unique usernames, but with a desired public perception of a function, not an individual. There are countless examples, admissions, enquiries, complaints etc.

Now this can be achieved in some cases by creating a “Helpdesk” username and a “Helpdesk” mailbox and simply having the helpdesk staff log into this mailbox, but more often than not people turn to shared folder access or delegated permissions as a solution.

Anything we can do with Exchange and Outlook to solve this problem or address this need is a principle based security approach to a role based security requirement, and as such it usually requires a little thought from the user in question as to exactly what they are trying to achieve, and what the best way to do that would be.

I haven’t mentioned resource mailboxes where the mailbox calendar is used to establish free/busy information for meeting rooms or equipment. These are a bit more mainstream once you get your head around the fact that you now need a user object called “Meeting Room 1” that never actually log’s in, but that does have a shared calendar.

So, history lesson out of the way, what options do we have.

Granting Access to Shared Folders

Each mailbox contains a set of folders. Out of the box these include Calendar, Contacts, Drafts, Inbox, Junk Mail, Notes, Sent Items, Tasks and a few others, and obviously any folders that have been created by the mailbox owner or any plug-in or add-on components such as AV, Archive, or Anti-Spam software.

Each one of these folders is an object, owned by the mailbox owner. You can verify this by opening DSA.MSC, and selecting a user object with a mailbox. Right click and select Properties, go to the Exchange Advanced tab and select Mailbox Rights...

The mailbox rights include the SELF object, which is allowed full mailbox access

As the owner of the mailbox, you have the rights to share any of your folders, exactly the same way you would folders on your local hard disk.

You simply right click, Properties and use the Permissions tab to assign granular permissions to the folder.

There are several levels of pre-defined permissions you can set most of which are pretty self explanatory, but just note for the minute the following, Reviewer, Author, Editor. I’ll come back to these in a second.

What you are doing here is modified the ACE’s on your folder ACL’s to grant or deny users granular access to your mailbox folders based on their Security Identifiers.

Gaining Access to Shared Folders

There are two ways to gain access to folders once permissions have been granted to you. You either use File, Open, Other User’s Folder... if you want temporary access one of the six default folders which are Calendar, Contacts, Inbox, Journal, Notes, or Tasks.

Or you Open these additional mailboxes: from the Advanced tab, of the mailbox properties. You either get to this by right clicking on the mailbox root and selecting Properties for “Mailbox – Declan Conroy“, or you go thru the more long winded Tools, Account Settings..., Change..., More Settings..., Advanced

In order to be able to open an additional mailbox, you must have a minimum of reviewer access to the top level mailbox folder itself. This differs from File, Open, Other User’s Folder... where you are opening a folder within the mailbox to which you have been granted specific access.

What have we got so far.

For varying levels of granular permission, ranging from Reviewer thru to Owner, we can grant access to any folder within any mailbox, and have other users either temporarily open these individual folders (File, Open, Other User’s Folder...) or automatically open all of the shared folders to which they have permissions each time they open outlook using Open these additional mailboxes:

Delegate Access, Send-on-Behalf-of

So what is delegate access all about then?

Well, like it says on the Delegate Access tab from the Tools, Options... menu delegate access goes a step further than shared access, in that it grants send on behalf rights to the mailbox delegate. To be more accurate, it populates the publicDelegates attribute of the AD user object for the user who is delegating access to his/her mailbox.

You can verify this by looking up your user object in ADSIEDIT.MSC under the domain partition and right clicking it to get Properties. Tick the box to Show only attributes that have values and scroll down till you see publicDelegates.

This attribute is a multi-valued attribute that will contain the DN of every object you have assigned delegate access to.

Interesting point to note, the following attribute called publicDelegatesBL is a list of the DN’s of all the user objects who have delegated access to you.

You can use the following AD search to list all of the users in a domain who have delegated mailbox access and the mailboxes they have delegated access to: (The line has been word wrapped for readability)

LDIFDE.EXE -F DELEGATES.TXT -D "DC=DOMAIN,DC=COM"
-L NAME,PUBLICDELEGATES,PUBLICDELEGATESBL
-R "(|(PUBLICDELEGATES=*)(PUBLICDELEGATESBL))"

The publicDeleagetes attribute values can been seen in AD users and computers. Open DSA.MSC and find your user object. Right click and select Properties. Clicking on the Exchange General tab and open Delivery Options... Your delegates are listed under Grant this permission to: in the send of behalf box

A blog for another day, but don’t modify the user attribute properties using ADSIEDIT.MSC, bad things will happen....

OK, so back to Outlook. There are a couple of things here. There are only six folders listed by the delegate access wizard when you click Tools, Options ..., Delegates, Add. These are the same six folders you can gain access to using File, Open, Other User’s Folder...

There are only three levels of permissions you can grant directly via the delegate access wizard, Reviewer, Author and Editor. You will find many references to shared folder permissions on the Internet and Microsoft documentation that list and describe the various granular permissions that can be assigned, and most of these descriptions are accompanied by the rather vague references that this permission (Does not apply to delegates.) All this simply means is that you can only assign one of the three levels of permissions above via the delegate access wizard.

It doesn’t quite hang together

I have to be honest I don’t think the various options hang together well. Here’s why.

When you use the delegate access wizard, you are limited to six folders, and three permissions. These folders don’t include the top level mailbox, and the permissions don’t include Owner.

So your mailbox delegate can gain temporary access to your inbox, and can send mail on your behalf. (Interesting point to note here, if you grant no access to any of the folders via the delegate access wizard (set them all to None) the publicDelegates attribute is still populated, but the user you have delegated to has no access to your inbox, so even though they are authorised to send mail on your behalf they have no access to your inbox to actually do it. That’s just odd.

The answer really is on the tab, if you use the delegate access wizard, you are granting people the authority to send on your behalf!

What delegate access doesn’t do is allow your delegates to Open these additional mailboxes: as the top level mailbox folder permissions aren’t modified by the wizard.

Now in most cases I’ve come across, if you are granting send on behalf rights to somebody they will usually want to spend a lot of time in your Inbox, and as such temporary access using File, Open, Other User’s Folder... just isn’t the best way to work.

So to craft a working solution you have to assign delegate access for everybody that you may want to send mail on your behalf in order to populate the publicDelegates attribute, and then manually, granularly set folder permissions at the top level mailbox folder, and also Sent Items and any other folder that is not one of the default six, so that they can automatically and permanently access your mailbox thru Open these additional mailboxes:

Worth pointing out, if you set the top level mailbox folder permissions to Owner for one of your delegates he or she can then Open these additional mailboxes: and once logged in to his/her own mailbox have the option to right click on your mailbox in his/her profile folder list, access the permissions tab and modify folder permissions for other people.

Finally, unless you grant a minimum of Reviewer access at the top level “Mailbox -” to your delegates there is no point in granting any shared access permissions to any mailbox folder other than the default six. Without top level Reviewer permissions the only access to the shared or delegated folders is via File, Open, Other User’s Folder... and File, Open, Other User’s Folder... is hardcoded to only list these six folders.

Making some kind of sense of the options

To make some sense of the various options available in shared folder access, delegate mailbox access, open other user’s folder, open additional mailboxes, and send-on-behalf-of, here’s what we came up with.

· Resource mailboxes. If you want to use a resource mailbox, either for a meeting room, or a piece of equipment you simply need to create the resource mailbox and change the default permissions on the Calendar to something like Author, which allows people to create and read appointments, and delete their own appointments.

Anybody can now use File, Open, Other User’s Folder... or use a Free/Busy search to book an appointment in the diary for the resource mailbox. (There are two other options for shared calendars in Exchange 2003, which are Public Folders and Moderated Public Folders, but if you want to use automatic free/busy searches you need to use the resource mailbox)

You may want to restrict the Author permissions based on security group membership, and the options here are limitless, in terms of granting certain groups Author and other groups Reviewer etc

· Shared or departmental calendars. If you want to more easily use a resource mailbox as a departmental or shared calendar you then need to add Reviewer permission for the department security group to the top level “Mailbox -” folder so that department staff can Open these additional mailboxes: and have more permanent access to the calendar. This also applies to any other folder, but I can’t think of any folder apart from Calendar that really requires a mailbox of its own.

· Delegated mailboxes. There isn’t really any call for shared access to a mailbox unless you need to send mail on behalf of somebody. I’m sure there are situations out there where this is being done, but from a business logic or work flow perspective if you can’t respond to the mail that’s arriving what use is reading it?

In this situation you have to jump thru all the hoops. Delegate the access to the default six, then manually modify the permissions on any remaining folders including the top level “Mailbox -” so that your delegates can use the Open these additional mailboxes:

Finally

It’s worth pointing out that the delegate access wizard doesn’t do any kind of enumeration or checking of permissions.

You can get into trouble very fast. In a situation where you remove delegate access using the wizard, users can still File, Open, Other User’s Folder... or even Open these additional mailboxes: because the wizard does not undo any granular permissions that you have set manually on other folders.

You also will come across a situation where you reapply permissions via the wizard and they will overwrite any you had manually specified at the folder level.

There is a lot you can do, but most of the complexity is in deciding what solution fits what purpose and then maintaining some control over it.

I haven’t touched on Send-as, I’ve restricted the options to those at the users fingertips and in my case Send-as violates an e-mail policy that mandates all outbound e-mail must be attributable to the sender, and I’m sure there are dozens of questions about specifics that I haven’t covered but hopefully my many hours of experimentation and Googling will save somebody else some time.

Declan

Dedicating an SMTP bridgehead server

2007-09-14T07:49:00Z

I got a panic stricken phone call from a customer at 18:00 on Wednesday evening.

They had over 10,000 inbound messages stuck on the SMTP bridgehead server that were not being delivered to mailboxes. Outbound SMTP was working fine. Inbound wasn't.

We worked out that the last inbound message has been delivered at about 11:00 on Wednesday, so assuming the default, by 11:00 on Friday these messages would start to be NDR'd.

We had a bit of time, bit not a lot.

Oh, and I was 200 miles away, on another site. It was going to be one of those nights.

So, first, what's changed. Well nothing, apparently. I don't believe that things ever just stop working, but trying to find anything that has changed is a fruitless excercise sometimes.

The SMTP bridgehead is in a DMZ, so we open up the firewall, and permit any any, but no luck.

We telnet to port 25 on the internal mailbox server, and sure enough we get a 220 ready.

We stop and restart the Default SMTP virtual server, and it takes a while, but it starts with absolutely no problem.

We increase diagnostic logging, and try again, and sure enough, no problem, no errors.

Time to get a little curious. SMTP seems perfectly happy.

We check the c:\program files\exchsrvr\mailroot\vsi 1\ directory structure. Badmail is empty, Queue is full.

This is good, it means if we can kick start SMTP the mail is not yet lost.

I ask them to check which Queue the message are stuck in. Its messages awaiting directory lookup....

This is the Active Directory, not DNS, but just to be safe we ping the internal server using it's IP, and name, and FQDN, and yep no problem.

This is one of the servers I built years ago, so I know the Support Tools are installed.

We run LDP, and we connect and bind to AD. I can even run eventvwr and open the event logs on the internal server, so we know RPC is working fine.

In a fit of hunger fueled desperation, I suggested we cross a bridge we would have to cross at some point anyway, and reapply SP2.

This gives me time to eat :-) I ordered a pint of Stella, and a chicken, bacon, brie and cranberry burger.

I consulted the oracle. I googled for messages awaiting directory lookup.

I came up with these very usefull links...

Messages awaiting directory lookup. This queue contains messages with recipient addresses that have not been resolved against the Active Directory. Messages are also held in this queue while distribution lists are expanded

Article ID: 251746 How to troubleshoot messages that remain in the "Messages awaiting directory lookup" queue in Exchange Server 2003 and in Exchange 2000 Server

Article ID: 884996 Messages remain in the "Messages awaiting directory lookup" SMTP queue in Exchange Server 2003 or in Exchange 2000

Article ID: 308350 Problematic message may be continually retried and may hold up other messages in connection queue

I also found this extremely detailed blog entry

http://msexchangeteam.com/archive/2006/06/23/428114.aspx

I sent all of this to my customer. When I rang them back, after going thru all of the troubleshooting steps, we were still no closer. As seems always to be the way, we had found twenty references to a cause for this problem, but we had the twenty first!

In the mean time my customer had run the ExTRA against the server. This had told him that there was a corrupt log file, and he was trying to recover the IS off tape.

Now this was brilliant news.

I asked him to open AD users and computers, DSA.MSC

Right click the domain object, and choose Find...

With the default of Users, Contacts, and Groups switch to the Advanced tab

Drop down Field to User and select Exchange Home Server

Change the Condition: filter to Ends with and in the Value: enter the name of the SMTP bridgehead.

In a situation where the IS isn't up, and you can't simply expand the Mailbox database, and check the Mailboxes container, this is the only way to determine how many mailboxes are actually on the server.

There were none listed.

There are actually three mailboxes, SystemMailbox, System Attendant and SMTP, but these are all system mailboxes, they don't show up in the AD LDAP search above, because the mailboxes don't belong to AD users.

Now, the SMTP mailbox is used for generating NDRs.

If you run SMTP, and you want to generate NDR's you need a mailbox database to be mounted, which is one of the reasons I don't like to run SMTP on an FE server. I like to keep the FE server purely for HTTP/S traffic, but I digress...

This was brilliant news.

I explained about message dial tone recovery.

Net Stop MSExchangeIS

Rename all of the MDBDATA directories to MDBOLD, this way we still have the original database and logs if we need them.

Create a new MDBDATA directory anywhere we have renamed the old one.

Net Start MSExchangeIS

Right click Mailbox store, and Mount Store

Ignore the error and press OK.

Presto, 10,000 messages started to clear instantly.

Now what's the lesson we learn here.

We would not have had this dial tone option, if the SMTP mailbox was on a mailbox store database with live users...

The quick win option was only available here, because the SMTP mailbox was in it's own storage group, there was nothing else that shared the transaction logs, so we had the option to sacrafice them and any content in the database.

How much does a dedicated SMTP bridgehead server cost.

How much even would a storage group dedicated to SMTP costs.

Compare that to the price of 10,000 lost e-mail...

My burger was cold...

How much did that cost me, £8.75...