luai7

Hi,

We have Team Foundation Server installed in our premises. We arranged with all the people working on TFS to make sure that their connection to TFS is closed before they leave work, so that to enable our SQL Server Full Backup Schedule to work correctly at around 03:00 AM.

Our questions are:

1. Do we have to back up the Reporting Services Encryption Key every night as well Or we do that only once in a life time

2. Is there anything else to backing up TFS - in order to gurantee successfull recovery in the case of a hardware crash - other than backing up TFS databases and RPT Service Enc. Key

3. Can we still reuse the files that exist in users' worksapces in the case of a TFS crash I mean after we recover/restore TFS, can we also recover non checked-In files from these users' workspaces What are the best practices in dealing with workspaces and their existing folder/files in such a scenario

Thanks in advance



Re: Team Foundation Server - Administration Team Foudation Server Scheduled Backup + Disaster Recovery Scenarios

James Manning - MSFT

1) the key only changes at time of install - I usually recommend backing it up right after (re)installing SQL RS.

2) That's it - you can also confirm on the "backing up tfs" docs @ http://msdn2.microsoft.com/en-us/library/ms253080(VS.80).aspx

3) Yes, part of the data backed up and recovered includes workspaces and their pending changes (tbl_PendingChange in TfsVersionControl). When the recovery happens, the server is right back to where it was at that point in time (and up to the minute of the crash assuming you're playing back SQL logs), so all your pending changes are just as valid as they were before the crash.






Re: Team Foundation Server - Administration Team Foudation Server Scheduled Backup + Disaster Recovery Scenarios

luai7

Hi James,

Thanks a lot for your help.

Regarding point 3, I think in our case the recovery will be valid up to the point of the backup taken @ 03:00 AM, that's to say if there is a hardware failure, we have only this snapshot of the data. In this case, all work performed by users on their PC workspaces is not part of the backup! That's why I raised this point. How does this change our recovery scenario

Another question please:

As part of our system administrator responsibilities, he has to perform a monthly restoration test (Usually at the end of each month). The whole idea here, is that we want to make sure that the monthly backups we have are indeed valid and we can count on them in case of a hardware failure. The point raised here is: If the system administrator is to perform this restoration test using the very last full backup he has, obviously he will do that on a server with a different name (Because the real production server is still online). What he will do, is after restoring the backup, the project managers will connect to this new TFS server and try to validate the various projects artifacts. Once this is done, the system adminstrator will dismantel this temporary restoration server.

What are the best practices for performing such a restoration test, taking into consideration that the server name is different and the original production server offcourse is still online Can we still use the "How to: Restore Team Foundation Server Data" document in MSDN because I found that that is slightly irrelevant to our restoration test case What do you think

[I forgot to say that we use a single server deployment strategy. So everything (Application, Data,... etc) is in the same server].

 

Thanks agian

- Luai

 





Re: Team Foundation Server - Administration Team Foudation Server Scheduled Backup + Disaster Recovery Scenarios

James Manning - MSFT

1) Backing up the contents of what's on each of the PC's can be handled in various ways. If you're getting your users to disconnect each night anyway (so you're already putting some burden on them), an "ideal" solution would be to have them shelve their changes each night to make sure it's included in the full backup. Then, even if their PC dies at the same time as the TFS server, they (or someone else) can unshelve their changes and get all the edits they have on their disk at the time of doing the shelve.

2) Since you're using a different machine name, we consider this a "restore-based move" scenario, which is documented here: http://msdn2.microsoft.com/en-us/library/ms252516(VS.80).aspx






Re: Team Foundation Server - Administration Team Foudation Server Scheduled Backup + Disaster Recovery Scenarios

luai7

Thanks a lot James.

I think shelving is indeed the solution for users temp. work backup.

Thanks also for the link concerning restoring TFS backup to another server (I missed out this document, I don't know why )

- Luai.





Re: Team Foundation Server - Administration Team Foudation Server Scheduled Backup + Disaster Recovery Scenarios

James Manning - MSFT

Great.

Also, as I alluded to before, you really want to include transaction logs as part of what you're backing up so that you can restore much farther along than just the 3am full backup.

Also note that there's no need for an offline/cold backup - our internal TFS server gets backed up every night while it's still being actively used.

Some links that may be helpful:






Re: Team Foundation Server - Administration Team Foudation Server Scheduled Backup + Disaster Recovery Scenarios

luai7

Hi James,

Thanks a lot for your continous support

Offcourse, we are considering transaction logs which will be helpful to recover up to the point of a crash. But, here I am assuming the worst possible scenario, in which the whole hardware including the hard disk is damaged . In this case, we have to go back to the last backup from tape/DVD.

Concerning your remark regarding offline COLD backup at night, it is very interesting to know that TFS does that automatically! Because I didn't know that before. Where is the backup stored Is it a Full backup

Also, if this happens while users are working on the system, wouldn't that affect the integrity of the database backup (Because of the time lag between the various database backups)   Or is this handled intelligently by TFS  

 

- Luai.





Re: Team Foundation Server - Administration Team Foudation Server Scheduled Backup + Disaster Recovery Scenarios

James Manning - MSFT

TFS doesn't do its own backup - when I said "internal", I meant inside Microsoft :)

I was stating that when our internal operations teams do their nightly backup of our TFS servers, they do an online backup - they don't take the database offline and do a cold backup.

Your transaction logs can be kept on a separate drive and they can also be replicated or shipped to another database.






Re: Team Foundation Server - Administration Team Foudation Server Scheduled Backup + Disaster Recovery Scenarios

Dipesh Bhattarai

Reading this thread was very informative and similar to what I have done. Ok, we have successfully restored TFS from one hardware configuration to another. Single server environment within the same domain.

As one of the test to validate my restore..We did a get latest of the SOURCE source-control to a folder just before the backup(offline mode). After restore onto another server, I performed the get latest of the DESTINATION source control again. We did compare these folders, to my surprise I find that some files are missing. I noticed that folder at eight level deep from the root had this issue. Example. a folder at the source has four files while at the restore it has only one file.

We have a huge source control and manual update is not an option. Any insight will be very helpful.






Re: Team Foundation Server - Administration Team Foudation Server Scheduled Backup + Disaster Recovery Scenarios

James Manning - MSFT

If you're in VS and open source control explorer on the destination, do the files exist in the tree Since you mention that the missing files are deep in the tree, it may be the case that those files couldn't be fetched in during your second get operation because the local paths would have been too long for those. Moving the root of that second workspace to a shorter path should help get around that if so.




Re: Team Foundation Server - Administration Team Foudation Server Scheduled Backup + Disaster Recovery Scenarios

luai7

Dear James,

Is there a limit on Workspace tree levels ! What exactly is it !

Any other limitations !

Please advise.





Re: Team Foundation Server - Administration Team Foudation Server Scheduled Backup + Disaster Recovery Scenarios

James Manning - MSFT

Not explicitly, but Windows has a max local path of 260 and we keep the same path limit on the server of 260. What this means in practice is that you can easily construct workspaces where we can't get all of the files because some of these will exceed the max local path.

Unfortunately, due to a bug in V1 you don't get an error message about those particular files, they're just silently not downloaded.






Re: Team Foundation Server - Administration Team Foudation Server Scheduled Backup + Disaster Recovery Scenarios

Dipesh Bhattarai

I figured out the issue but I dont have a workaround yet. Its the power tools compare function that was leading me to this situation. Let me explain.

We had a folder at the 8th level, I always do a get latest to local path. So the folder got created in the local path. Next month with the new code promotion, the source code got changed and the folder and its files were no more part of the monthly release. As such we got rid of the folder in the source control. BAAM ..not on the local path.

We use compare function on Power tools to check in the difference in the monthly release. When this tool was used with checked item that exist exclusively in the local path, we did not see any extra files/folders. so we called it good. But it looks like this tool is not capable of true comparision at lower levels.

My issue had happened becuase of the ghost folder/files that remain in the source local path and not in the destination path. Anyways..now the question changes direction. I believe Power tools are not yet supported product. Any direction from here.






Re: Team Foundation Server - Administration Team Foudation Server Scheduled Backup + Disaster Recovery Scenarios

Dipesh Bhattarai

For the above issues, I created a new thread. http://forums.microsoft.com/MSDN/ShowPost.aspx PostID=1279953&SiteID=1 on Power Toys Section.

Going back to Team Foudation Server Scheduled Backup + Disaster Recovery Scenarios: I have a question in general.

Can we completely automate the TFS restore process on the separate server. Having to go through manual steps as mentioned in the http://msdn2.microsoft.com/en-us/library/ms404869(VS.80).aspx is ok, but an automated restore everynight on DR site would be owesome.






Re: Team Foundation Server - Administration Team Foudation Server Scheduled Backup + Disaster Recovery Scenarios

James Manning - MSFT

It's *possible*, as I believe at least one customer has done exactly that, but there's nothing we officially support for it.