Photos (Part 2) - Exchange / Active Directory/ Office 365 Photo Importer

With Part 1 of this series I introduced you to a tool that will allow you to convert photos into a format suitable for importing into Active Directory, Exchange, or Exchange Online. These photos are used across the whole Office product line, including Lync/Skype for Business, Exchange, Sharepoint, Office 365, etc. After creating these images, the next step is to import them into either Active Directory and/or Exchange 2013. The final outcome is having glorious photos appear in your Office applications!

Look! Pretty Photos!

So once again, to try and save everyone a lot of pain, I've made a tool that will hopefully make importing these images a breeze.

Exchange / AD / O365 Photo Importer

The aim of this tool is to be simple and flexible as possible to take the pain out of importing photos in any scenario.

• View previously uploaded Exchange HD images, Active Directory images, and Office 365 Exchange Online HD images.
• Import Exchange HD images, Active Directory images, and Office 365 Exchange Online HD images.
• Remove images from Active Directory or Exchange HD photo for any user.
• Downscale previously imported Exchange HD images to 96x96 sized images in Active Directory by pressing the “Use Existing HD” button. This replaces the 64x64 image that Exchange auto-uploads to AD when you do a HD image import. This button can be handy if you no longer have the source image on hand and want to quickly upgrade the resolution of your AD photos.
• Automatic detection of On Premises or Office 365. The system type that is detected when the tool boots and the system type will be shown in the top right hand side of the interface. Note: The system type will affect the naming convention used for the user names in the tool. On Premises will use the SAMAccountName and Office 365 will use the Alias/Username of the user.
• Automatic resizing of images to 96x96 before they are being imported into Active Directory. This stops you from uploading unnecessarily large images into Active Directory. (ie. if you open a 4MB picture into the tool and try to import it into AD, the tool will convert the image to 96x96 before uploading it)
• The View Web Image button will open a browser connection to the 648x648 sized version of the image. This can be useful if you want to download a copy of the HD image as a backup.

Requirements:

• The script is supported on Powershell Version 3.0 and above. So if you're running Windows 7 you will need to make sure you've upgraded your Powershell version to at least Version 3.0.
• Drag and Drop only works when the filesystem and Powershell session have the same security level. So if you're running Powershell with Administrator privledges (ie. Run as Administrator), whilst you are logged into the machine as a different username, the Drag and Drop function will not work. To fix this just run the Powershell with the privileges of the user you logged in as (as long as you have the correct AD and Exchange permissions) and it will work.
• In order to set Exchange HD photos for On Premises or O365 the user that is running the Powershell session will need to have permissions to run the Set-UserPhoto command. The build in RBAC roles that support this command include Organization Manager, Recipient Management and Help Desk. To set Active Directory photos the user will need permissions to run the Set-AdUser command.

Version 1.01 Update (15/5/2015)

• Corrected issue with the tool on Powershell version 4. Removed "-ErrorVariable" flag from script because it was causing "language mode" errors on Powershell Version 4 with Remote Powershell connections.

Download Version 1.01:

Overview

The tool is designed to gracefully fall back to support whatever level of Powershell commands that are available to it. So if you were to run it on a machine that only has access to Active Directory commands it would only allow you to import Active Directory photos, and so on. Ideally you should have access to both Active Directory and Exchange commands for an on premise deployment. For Office 365 you will need to remotely connect to Exchange Online (see the next section for details of how to do this).

There are a few things that you should understand about how the Powershell import commands work before using the tool. The Exchange import command (Set-UserPhoto) supports the importing of any sized file JPG file into the system. If the file is not square in shape then Exchange will do a “centre crop” (as explained in my previous post) on the image and convert it to 648x648 in size then import it into the user’s mailbox. At the same time as doing this Exchange will also import a 64x64 sized image into Active Directory.

The Active Directory import command (Set-ADUser -identity $name -Replace @{thumbnailPhoto=$photoBytes}) does not support the same fancy cropping and resizing capability as the Exchange command does. Instead it will import the raw bytes that it is presented into the thumbnailPhoto attribute in the Active Directory database. The thumbnailPhoto attribute will accept images of up to 100KB in size. However, it’s not recommended to import files that are that big into Active Directory as it can add a great deal of size to the database which can result in much larger amounts of replication traffic between Domain Controllers. If you import an image into Active Directory and there isn't currently a HD image in exchange, then the AD image will also be displayed by the Get-UserPhoto command in exchange as well.

Tool Operation

Now that you have an overview of how the commands work we can go into some depth about how the tool works. The tool displays three images, the left most image (Input Image) is a preview of the image that you are going to import into the system. You can select this image by either dragging and dropping the image from your PC (see requirements section for more details of this), or by selecting the browse button under the Import Image section. The middle photo is the current Active Directory photofor the user highlighted in the Select User dropdown box. The rightmost photo is the current Exchange HD photo for the user highlighted in the Select User dropdown box. If the user does not have images in either of these locations then a generic missing photo image will be displayed by the tool. If the tool has not been able to access the necessary commands to get to the photo a “Not Accessible” message will be displayed. If the user does not have a Mailbox a "No Mailbox" image will be displayed.

When using the tool you may choose to import a single user photo or you might want to import a folder full of images. The tool will allow you to do both by selecting the Import Folder or Import Image checkboxes respectively. When the tool opens a file it will automatically select the import option appropriate for the image size. If the photo is 96x96 or smaller, then the Active Directory import check box (Replace 96x96) will be selected. However, if the image is larger than 96x96 then the Active Directory (Replace 96x96) and Exchange (Replace 648x648) import checkboxes will both be selected. In the case that both of the checkboxes are selected the tool will first import the image into Exchange (which will automatically import a 64x64 image into Active Directory) and then the tool will resize and import a 96x96 version of the image into Active Directory (ie. over the top of the smaller 64x64 image). 

When you select to import a folder you have the choice of seeing each image as it is imported and selecting whether you want to import the image or not. This by default is the operation of the tool, however, you may wish to import all of the photos without confirming each file. This is done by unticking the Confirm Import checkbox. When importing folders, the files within the folder need to start with a name that matches the user name in Select User dropdown box (which is the user's SAMAccountName from Active Directory for On Premises, or the Alias/Username from Office 365). The tool will allow you to have extra information in the file name, however, additional information must be separated from the user's name by a space character (space is used because it can’t be used in a SAMAccount name or Alias/Usernames in Office 365). For example, you may have a file named “John.Smith 648x648.jpg” which the tool will import for a user with the SAMAccount name of “John.Smith”. However, you cannot have the name “John.Smith648x648.jpg” because it doesn't have a space character between the name and additional test which means it isn't an exact match for the John.Smith user in the dropdown box. The important point to take from this is that you need to be precise in the naming of your images for a Bulk import. In summary:

On Premisis File Naming: SAMAccountName

Office 365 File Naming: Alias/Username

Note: Image file names must begin with the user's name following the convention above and must be divided from any other text in the file name by a space character.

The file naming for a bulk import is your most important job! After doing this you can kick back and let the tool do its work.

Importing Photos into Office 365

The tool has been designed to support importing HD photos into Exchange Online. However, the commands used to connect to Office 365 via Powershell may be slightly different than what you usually use. So Office 365 admins - pay careful attention to this section!

Create an O365 session:

$Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://outlook.office365.com/powershell-liveid/?proxymethod=rps -Credential $cred -Authentication Basic -AllowRedirection

Note: You must include the “?proxymethod=rps” part of the URI in order to import images into Exchange Online. If you don’t do this, the import will fail with a (413) Request Entity Too Large error.

Enter O365 credentials:

Import the session into Powershell:

Import-PSSession $Session

After you have done this, you can run the script in the same Powershell window and import photos.

Limits of Office 365

There does appear to be a maximum image file size that you can upload to O365. However, it seems to be more of a practical size limit rather than an enforced one. I have uploaded images up to about 6MB in size before, however this is a very slow process and you can run into random errors during the process. It can also take a long time (5-10 minutes) to import very large images directly into the Exchange Online server. When the photo has been uploaded the Exchange server will resize the image before saving it to the user’s mailbox. For the aforementioned reasons I would recommend reducing the size of this photo before trying to upload to Exchange Online. My suggestion is that you use my Image Creator Tool to reduce the size of your photos to 648x648 before you try to upload them.

I have found that Office 365 doesn't like you uploading photos for the same user multiple times in a row (ie. a bulk import with many photos for the same user name). If you do this you will get errors back from O365. However, if you are bulk uploading to different users it seems to work fine. If you start experiencing random errors when trying to upload or get images from Office 365 try disconnecting the current session and open a fresh Powershell session.

On occasion when uploading an image to O365 the upload process will work successfully (no errors in the PS window), however, the image will not be available to be viewed immediately after a successful upload. Majority of the time this is not the case but occasionally it will take about 10 seconds before the photo can be viewed. In this time the tool will display the generic no photo silhouette picture. If this happens, be patient and refresh the user photo by reselecting the user from the 'Select User' drop down box. 

The Wrap Up

Well, that’s photos in the bag! You should now be able to convert and create images as well as import them into an On Premises system or an Office 365 tenant. Hopefully this makes your life much easier. Let me know if you have feedback and enjoy!

Read more →

The Case of the Lync 2013 Edge Server Centralised Logging Ports

If you have ever done any port testing on a Lync 2013 Edge server you may have noticed that the external interface of the Edge server had ports in the 50001-50003 range open and listening for TCP connections. Usually this is not the case in the 50000-59999 media range because these ports are only opened for short periods of time when media ports have been allocated by the Edge server for active calls. The Media Relay Service on the Edge was designed this way for security purposes, so that the 50000 range could be opened on external firewalls without posing a significant security threat. So what are the undocumented 50001-50003 ports facing externally? Well, they are actually the Centralised Logging Service and they appear to be facing externally for no reason other than they are bound to port 0.0.0.0. The netstat command show this:

ClsAgent.exe Ports

The netstat output shown above shows that the ClsAgent.exe service is listening on the 50001, 50002 and 50003 ports on the IP Address “0.0.0.0”. This means that the service has not bound to any specific interface and as a result will listen on all interfaces. I image this is a design issue with the CLSAgent because it was originally designed to run on internal Lync servers that only had one interface and so binding to a specific port wasn’t a requirement at the time of designing the software. I’m happy to be told different by someone at Microsoft though…

As mentioned earlier, the CLS port range falls into the TCP 50000-59999 range, which is also legitimately used for Edge Media Relay service. So it is included on the list of external port ranges that may be open on the external firewall. I say may be opened because this range of TCP ports does not need to be opened inbound (as per the guidance from Microsoft) unless you are federating to OCS 2007, or in the more complex scenario when you are using NATing with DNSLB on the external edge and your firewalls do not support hairpinning (ie. traffic coming from one Edge server’s NATed external public IP address back in to another Edge servers NATed External Public IP address) of media between multiple edge servers in the same pool. Another legitimate reason for this is when you want an optimised media path that does not require tunnelling via port 443/3478 to get to the required 50000 range media port. These scenarios were explained in a great amount of depth by Bryan Nyce and Thomas Binder at Lync Conference 2014. I suggest you watch these videos several times if you don’t understand what I’m talking about here.

'So what percentage of companies actually open the 50000 range of ports to the internet?' I hear you asking… Well, I also wondered this, so I decided that I might do some research and find out. I tested approximately 250 Edge servers of some of the largest organisations in the world and found that approximately a quarter of them have the 50000 range open (with CLS ports showing). So quite a large number of organisations currently have this issue.

The actual security implications of having these ports open on the Internet is not fully known at this stage. The existing ClsController.exe application and CsCls Powershell commands supplied with Lync do not allow the user to connect to servers outside of the Lync pools within their installation. So it's certainly not the case that you can use them to randomly connect to other organisations' Edge servers and start logging service. It's my understanding that Microsoft is aware of this issue and have not yet done anything to change the behaviour, so they obviously have deemed it low risk. In my opinion though, the reduction of attack surface is always a good idea for internet facing services. So my recommendation is to block these ports because they serve no practical function externally.

A Work Around

I have written a Powershell script that will block the CLS service ports on selected interfaces of your Edge server. This will function as a work around until Microsoft decides to formally change this behaviour in the product.

When you run the script it will display a list of IP Addresses on the server that you can choose to block the CLS service on. You simply need to enter the number of the interfaces in the list that you would like to block access to the CLS ports on. The script will then automatically create a new firewall rule to block TCP ports 50001-50003 inbound on the selected IP Address to the CLSAgent service on the machine. Follow this process for all externally facing Edge IP Addresses (ie. Access Edge, AV Edge and Conferencing Edge IP Addresses).

BlockCLSExternalEdgePorts1.00

The rules that are added can be seen in the Advanced settings of Windows Firewall in the system Control Panel:

Once you have done this for all of your Internet facing Edge IP Addresses you can rest easy: your Edge is now as safe as you previously thought it was…

Download Version 1.00:

Note: Run Powershell as Administrator when running the script.

The Wrap Up

Security is important and we all need to try to understand what we are asking when we tell the firewall team to open internet facing firewall ports. Whether you are deploying a new Edge server or already have an Edge server out in the wild, then I suggest you implement my firewall work around. Enjoy and see you next time. Ciao!

Read more →

Photos (Part 1) - Lync / Skype for Business Photo Editor

Happy New Year! Hang on, it’s February… Time seems to have got away on me since I started this holiday project that turned out to be more complex than I had initially expected. What was originally going to be a simple photo resizer tool for Active Directory, Exchange and Lync/Skype for Business, turned into more of a complete photo editor including face recognition based cropping, image filtering and processing, written entirely in custom Powershell code… What can I say, sometimes I can’t help myself…

So let’s begin by discussing the problem: Lync and Skype for Business clients are used for communicating, and the best experience for this is when you are able to easily see the people that you’re communicating with. So within the Lync and Skype for Business client UI there are many places where photos of users are shown. This is great because it makes the software much more personal and approachable to users... and ever since Bill Gates' mug shot silhouette was replaced as being the default missing photo image from Lync 2010 (yes, is really was, see below!) to the generic Lync 2013 oval head silhouette, a photoless client tends to look a bit boring.

Image Reference

So for the best user experience we really should be using profile photos for users in Lync and Skype for Business… but what does that mean for us Lync Administrators? Well, usually it means that you are going to get given thousands of random image files, in god knows what format, and at who knows what quality, that you will be expected to magic into the system and have displayed for all to see. So let us put our magicians' hats on and I will help you with Part 1 of this trick… Turning a mess of image files into something that will work when imported into Active Directory and/or Exchange for display within Lync or Skype for Business…

Lync / Skype for Business Photo Editor

You will know by now, if you’ve ever read my blog, that a custom written Powershell tool will always provide the solution to our problems! I give you the Lync / Skype for Business Photo Editor Tool!

Features:

• Zero installation.
• Signed Powershell Script.
• Bulk conversion of a folder full of images files.
• Custom coded image processing!
• Manual editing and cropping of individual files. Simply Drag and Drop an image into the Picture Box area and then start selecting the crop box size and position you would like your output files to be based on. Set filter options and preview their effects on the image using the “Preview Filter” button. The size of the image crop box is shown under the scaling tools to allow you to know if you are cropping to a size smaller than image you are wanting to output (to avoid accidently upwards rescaling of the images).
• Accepts input files in the following formats: ".jpg", ".jpeg", ".gif", ".png", ".bmp" and ".tif”. All files get converted to “.jpg” format so they can be easily imported into Exchange/AD.
• Smart Crop / Centre Crop modes – By ticking this box (default) my Smart Cropping algorithm will be used to discover the subject's face and crop appropriately. The Margin setting is used in conjunction with Smart Crop to determine how loose or tight the framing will be around the subject. If unticked, a simple Centre Crop Method will be used. See the Smart Crop section for more details.
• Filters! Since Instagram became such a big hit, filters have become a must-have for all successful software projects. So why should this one be any different? Photo filters include: Colourise (Blue, Red, Green, Yellow, Orange, Pink, Purple), Contrast (Reduce Contrast, Light Contrast Boost, Medium Contrast Boost, Mega Contrast Boost, Ultra Contrast Boost), Brightness (Reduce Brightness, Light Brightness Boost, Medium Brightness Boost, Mega Brightness Boost, Ultra Contrast Boost), Effects (Old Film, Vignette, Light Leak, Vintage, and Slide Show).
• Output sizes. By default the tool will output 96x96 and 648x648 sized photos. These can be turned on or off using the checkboxes in the “Image Output Settings” area of the GUI. There is also the option to create custom sized photos by ticking the custom checkbox and selecting the pixel width/height of the photos to be outputted. It is generally recommended that 96x96 files are used for uploading to Active Directory and the 648 x 648 images are uploaded to Exchange 2013.
• Quality Control – The quality of the jpg images that the tool will output can be changed by reducing the Quality setting between 1-100. I suggest that you never actually reduce this in the process of creating the files that you are importing via Exchange, as the quality will be further reduced by Exchange as part of the import process.

Requirements:

• The script is supported on Powershell Version 3.0 and above. So if you're running Windows 7 you will need to make sure you've upgraded your Powershell version to at least Version 3.0.
• Drag and Drop only works when the filesystem and Powershell session have the same security level. So if you're running Powershell with Administrator privledges (ie. Run as Administrator) the Drag and Drop function will not work. To fix this just run the Powershell with regular privileges and it should be okay.

1.01 Update (23/4/2015):

• Added policing of the folder name to accept ending with a "\" or ending without a "\".
• Changed the output file name to use space (" ") instead of a minus ("-") character between the name and the image size ("648x648") to work with new Photo Importer Tool.

Download Version 1.01

Features - Smart Crop

One of the main problems with Exchange / AD / Lync images are that they must be square in shape, and digital cameras don’t usually take square photos (they are usually taken in a 4:3 or 3:2 ratio).  So as a result, after taking a photo it inevitably must be cropped before it can be used by Exchange or Active Directory as a Lync / Skype for Business photo. The way that Exchange handles this is to do (what I call) a Centre Crop on the image. This is where you crop to a square that is the width and height of the short side of the photo and then centre the square in the middle of the long edge of the photo (see the Centre Crop example image below). This works well when the image has been composed with the subject's face right in the centre of the frame. However, what if the person taking the photo decided to also include a large portion of the subject’s body in the frame? Or it’s one of the user’s favourite photos of them at the beach with the majority of frame consisting of landscape? Or what if, god forbid, the photographer decided to use their arts degree and frame the subject using the Golden Ratio or the Rule of Thirds… In these cases you can end up with a weird looking image if Centre Cropping is used.

Centre Crop Example

I realised all of this after starting this project and tried to think of a better way… like, what if I was to detect where the subject's face was within the image and then crop around it? That sounds like fun! So I started reading about facial recognition techniques and software. What I learnt was that there is no existing core Dot Net libraries that I could leverage in Powershell that would supply me with facial recognition. So I looked more broadly and discovered that there are a couple of open source projects that had Dot Net ports and could be used if imported and compiled into a Dot Net application… but I wanted this to be a pure Powershell implementation and not some bulky application! So I then dug deeper and started reading academic papers about different face and skin recognition methods that exist. After doing much prototyping and testing with these skin recognition concepts, I created my own skin thresholding algorithm in Powershell that was quite fast (something learned whilst doing this was that Powershell is slow at doing many mathematical operations, and especially slow when it comes to recasting variable types and object creation).

The end result of this absurd amount of work was an unassuming checkbox in the Bulk Import section of the tool’s GUI called “Smart Crop”. Smart Crop is used in two places in the tool, the first is when Bulk converting images from a folder.  The tool will try and locate using my skin recognition algorithm where the majority of skin is on the screen and then try to appropriately frame the around this location. The (3:2 ratio) photo below shows an example of what a better alignment for a centre cropped image would look like as a result of recognising where the face is in the image.

Face Aligned Centre Crop

As you can see in the above photo, the image gets cropped around the more important part of image rather than the centre of the image as seen in the earlier Centre Crop example.

So it’s pretty useful to be able to align a full width crop around the subject's face like this, however, Lync photos end up being reduced to very small sizes in most cases (96 x 96 pixels). So it would also be nice to be able to crop even tighter to the subject's face so that you can see it more clearly in your contacts list in Lync / Skype for Business. Below is an example of a better crop to use in Lync:  

Ideal Smart Crop for Lync

In order make the photo more usable with Lync/Skype for Business, I have tried to tune the algorithm to give a tight crop around the face of the subject in the photo. A pitfall of doing a tight crop like this, however, is that the cropping square should not be less than the size of the image that is being outputted (ie. the crop square being 200 x 200 in size and the output size being 648 x 648 in size). If this happens then the quality of the output photo will be significantly reduced due to the image being blown up. For this reason you should always try and use source images that are quite a bit larger than the largest size image you are trying to output. The tool has been designed to understand this issue and will always attempt to crop to at least the size of the output image file. This in some cases will result in a looser crop that you might expect around the subject's face, however, it is designed to maintain the quality of the output file.  

The “Margin” setting in the tool can also be used to tighten or loosen the Smart Crop frame around the subject's face. This setting ranges between 1-50, with a default of 25. The lower the value, the tighter the crop will be around the subject.

From the testing I've done so far I have found my Smart Crop detection algorithm works in the majority of cases (note: it does not work on greyscale images). However, it can have problems if there are background components in the image that fall into the same Luma and Chroma ranges as skin does. In these cases you can manually crop the individual files that were not detected accurately. In the end though I hope it saves you a bunch of time and effort!

Features - Filters

Photo filters are all the rage at the moment with every social media app in the world jumping on the bandwagon. However, the idea of filters in this application is not just a gimmick: it offers you the ability to do colour, brightness, and contrast correction to photos in order to give them more pop so they look their best when displayed in Lync/Skype for Business. When you are supplied photos by an organisation, it’s fairly likely that they were taken in a room somewhere with bad or at least uninspired lighting. As a result, all of the images can look washed out and flat.

Below is an example of a washed out image of a technology company CEO you may recognise. As you may be able to see, the photo on the left does not look very vibrant and comes across as quite bland (especially when reduced to 96 x 96 pixels in size). However, after applying a contrast and brightness boost filter to the image (as can be seen in the picture on the right) it looks much more dynamic.

  

The Lync / Skype for Business Photo Editor Tool gives you the option to Colourise, Contrast Reduce/Boost, and Brightness Reduce/Boost with various levels of intensity, and these settings can be chained to give you 175 different combinations to use!

In addition to the more subtle image quality and dynamics filtering capabilities I decided that I too couldn’t resist the challenge of implementing some grungy Instagram style filters. So if you have been using Instagram too much and feel the need to make your images look a little more vintage, try the Effects filters drop down box. Here’s some examples of my custom effects filters:

Will these filters ever be used for a Lync or Skype for Business deployment? Maybe not, but I had fun figuring out how to write the image processing code to generate them J

The Wrap Up

Well there you have it: my holiday project has finally made to a public release. You may have also noticed that this post is only Part 1 of a series. Indeed it is! Because now that you have a tool to easily create image files for Exchange and Active Directory, you will likely also need a tool for easily uploading those files to these systems. So Part 2 of this series will supply you with just such a tool… So keep an eye out for that one. Cheers, and enjoy!

Read more →

Sonus SBC 5k/SWe CDR Decoder

I was on a training course recently learning about the SBC5000/SWe series SBCs from Sonus as these are now supported with Microsoft Lync 2013! This range of SBCs are completely different from the SBC1000/2000 range that you may be used to in your previous Lync deployments. The reason for this difference is that the SBC1000/2000 range were originally designed around a completely different code base by the NET company which was later acquired by Sonus. 

The SBC5000 was  originally designed for large carrier deployments, however, a newer virtualised version of the SBC5000 called the SWe (Software Edition) starts to make these systems much more affordable and interesting for enterprise sized customers. During the training course we were introduced to an online tool that Sonus has for parsing individual Call Detail Records (CDR) to show you what each field in the record represents. This may not sound like much until you see the size of a SBC5000’s CDR record… Behold!

STOP,PRGSBC51,0x000174D700000001,3955074504,GMT,10/22/2014,01:11:19.7,0,4,37,10/22/2014,01:11:40.8,3,2076,16,VoIP,IP-TO-IP,DEFAULT,,,6731234507,,0,,0,,0,,RL7,1,PRGSBC51:TG07,192.168.229.118,192.168.228.237,TG07,,192.168.229.117:1024/10.128.176.185:5062,,192.168.229.117:1026/192.168.228.237:5048,213416,1036,176476,1025,0,,,0x00800000,,,,,2,"SIP,009258DD-F957-E411-8C61-342F2AB2DB43@10.128.176.185,<sip:PhonerLite@10.128.176.185>;tag=3367902045,<sip:6731234507@192.168.229.118>;tag=gK0c816f95,0,,,,sip:6731234507@192.168.229.118,,,,sip:PhonerLite@10.128.176.185:5060,sip:6731234507@192.168.229.118:5060,,,,1,BYE,,0,0,,0,0,,,,,,,,1,0,0,0,,,,,,,,0,,",12,12,0,5,,,0x0a,6731234507,1,1,,1,0,0,0,TG07,"SIP,786432_119514912@192.168.229.118,<sip:PhonerLite@192.168.229.118:5060>;tag=gK0c0170af,<sip:+16731234507@192.168.228.237:5060;user=phone>;tag=4fbfd5e7e6e006f0,0,,,,sip:+16731234507@192.168.228.237:5060;user=phone,,,,sip:PhonerLite@192.168.229.118:5060,sip:+16731234507@192.168.228.237:5060;transport=udp,,,,,BYE,16,0,0,,0,0,,,,,,,,1,0,0,0,,,,,,,,0,,",,110,,,1,1,,,2,P:2:1,P:2:1,10,0x000C0000,,,0,,,,,,0,,,,1,,,,,,,6,,,,,,1,1,1,1,,0,,,1,7,0,2076,1,,,,,192.168.229.118,10.128.176.185,2,16,8,,,,,,,,,0,,,TANDEM,,,10,211150,1025,178192,1036,0,0,0,35,20,,,,,,,13,1,,,,,,,,,,,,,,,,,,,,,0,9,,,,,,,,,,,,,,,"3,39,0,42",0,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,"01,audio,1,G711S,192.168.229.117:1024,10.128.176.185:5062,0:0:0:0,56,G711S,192.168.229.117:1026,192.168.228.237:5048,0:0:0:0,124","01,audio,1,1036,1025,213416,176476,0,0,1025,1036,211150,178192,0,0",0,,,

From this mess you might see how finding field 134 might take some time… So Sonus’ online tool is a very handy thing. The CDR records also contain a great deal of useful information about a call (kind of like Lync Monitoring Database records) such as Codecs Used, if transcoding took place, SBC routes used, reason for disconnection, etc. These records can be extremely useful when troubleshooting issues with the system.

Sonus Online CDR Decoder

I was thinking, though, that you don’t always have access to the internet when you’re working on a customer site, and often you might want to look through a whole file worth of records (instead of cutting and pasting individual records into the online tool). So I decided that it was time once again to get cracking on a Powershell tool for browsing SBC5000 CDR files … and after eleven thousand lines of parsing code I bring you:

Sonus SBC 5k/SWe CDR Decoder

Version 1.0:

• Import SBC5000 or SWe CDR files (ACT files).
• Left hand list view will display all of the records that are in the file.
• Right hand list view displays all of the fields in the selected records.
• Subfields (ie. fields that contain multiple pieces of information in them) are individually broken out and displayed in grey colour.
• Fields are decoded where possible with the decoded value of the field shown in brackets next to the field entry.

Version 1.01 (5/12/2014):

• Added some more enumerations.
• Fixed a few small parser bugs.
• Added a show as text button.

Download Version 1.01:

How to access SBC5000/SWe CDR files

CDR files are also called account files, and they can be easily downloaded from the system via Platform Manager. Simply access Platform Manager (ie. management interface IP on port 444) and go to the Logs -> Event Logs section and select ACT as the log type you wish to access. Then download the file with the Download button:

Then open the Sonus SBC 5k/SWe CDR Decoder tool in Powershell and select the “Browse…“ button, select the ACT file, and click the “Load” button to import the contents of the file. Now you’ll be able to browse the records in the left hand listview and each record you open will be displayed in the right hand listview.

The Wrap Up

At this stage there may not be many deployments of the Sonus SBC5000 or SWe in Lync deployments around the world. However, I can see the SWe being a highly flexible and tenantable virtualised SIP Trunk SBC in the future. So even if this tool doesn’t seem immediately useful it may someday become handy for troubleshooting your future SWe deployments!

Read more →

When Poodles Attack - Poodle Checker Tool

It’s not too often you get to be excited about a security threat. However, the POODLE security threat seems to put a smile on my face every time I see it written somewhere… Poodles are just so innocent and ridiculous looking to take seriously as a major threat. So in a bid to take this security issue more seriously, I have built a Powershell tool for remotely checking servers for having either SSL 2.0 or SSL 3.0 enabled on them.

More Detail on the POODLE threat

Here are some links that explain the POODLE threat in a little more detail:

• https://www.openssl.org/~bodo/ssl-poodle.pdf
• https://blog.mozilla.org/security/2014/10/14/the-poodle-attack-and-the-end-of-ssl-3-0/
• In relation to Lync and Exchange: http://masteringlync.com/2014/10/20/poodle-and-lync-server-2013/
• Microsoft’s Security Advisory: https://technet.microsoft.com/library/security/3009008
• Office 365 Statement: http://blogs.office.com/2014/10/29/protecting-ssl-3-0-vulnerability/

POODLE Checker Tool

• The tool will try and connect using SSL 2.0 and SSL 3.0 to any server FQDN/IP and port (multiple ports can be entered with a comma separating them) you enter.
• Press the Test button and it will check all the ports in the ports text box. The tool will report in the Powershell window which ports have SSL 2.0 and SSL 3.0 running on them.
• The tool will also visually display the results…
• Script is signed.

Update 1.01

• Added additional checking of TLS (1.0, 1.1, 1.2) protocols so you can better understand all the TLS connection options available on the server before deciding to disable SSL. 

Update 1.02 (16/2/2015)

• Added the ability to handle multiple comma separated IP Addresses/DNS Names.
• Added Cancel button to stop testing.
• Disabled text boxes during testing phase.
• Textboxes now stretch when resized.

Download Version 1.02:

Standard SSL Port Numbers

SSL can technically run on any port that you configure and application to use. However, the well-known port numbers for applications that use SSL (as defined by IANA, and IETF) are listed below:

Protocol	Port	Description
nsiiops	261	IIOP Name Service over TLS/SSL
https	443	http protocol over TLS/SSL
ddm-ssl	448	DDM-SSL
smtps	465	smtp protocol over TLS/SSL
nntps	563	nntp protocol over TLS/SSL
sshell	614	SSLshell
ldaps	636	ldap protocol over TLS/SSL
ftps-data	989	ftp protocol, data, over TLS/SSL
ftps	990	ftp, control, over TLS/SSL
telnets	992	telnet protocol over TLS/SSL
imaps	993	imap4 protocol over TLS/SSL
ircs	994	irc protocol over TLS/SSL
pop3s	995	pop3 protocol over TLS/SSL

Note: A listing of all IANA port assignments can currently be found at: http://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.txt

I have made the tool load all of these ports into the port text field by default.

Note: The documented attack vector for POODLE is described for HTTPS connections, and not necessarily for these other protocols. The tool checks all of these protocols to check if your server is still accepting SSL2/3 connections in order to determine if it's globally enabled (in Windows the registry key effects SSL across most applications). Also, additional attack vectors may be found for other protocols, so if your applications can support newer versions of TLS it is probably wise to turn these older versions of SSL anyway.

The Wrap Up

There you have it, short and sweet! I hope the tool is useful to you and helps you take security issues more seriously J

Let me know if you find any bugs or have any issues.

Read more →

Power Syslog Server

“My Kingdom for a free and simple syslog server!” – Anonymous System Administrator

So I don’t know about you, but I can’t remember how many times I have got to the point of having to troubleshoot an issue with a Sonus gateway and suddenly remembering I need a Syslog server to get logging out of the box. At this point I usually go and ask Google politely “Google, can you please point me in the direction of a free, and simple, syslog server that I can run without installing a bunch of malware and other rubbish on this nice customer’s server?” At this point Google usually responds “No, I cannot. However, here is a syslog server that requires you install SQL, IIS, and fifteen other dependent services as well as being crippled unless you pay $14.99 per month to a Russian guy name Vlad via this popup window that displays in the middle of the screen every 5 minutes. Also, here’s a Yahoo browser search bar for your trouble.”

This is not an ideal situation… So as usual, I just decided to build it myself. In doing this I sat down and thought about the things I wanted in a simple syslog server, and came up with this list:

• It needs to have no installation process, and leave no trace once removed from a server, as it will be run on customers' servers in a lot of cases.
• It needs to have a display where I can see the messages coming in in real time.
• The messages being displayed must be able to be paused and reviewed, so I can check if a specific event has happened yet.
• The messages window must be able to be cleared so that I can start fresh when trying to troubleshoot a fault.
• The syslog server needs to be able to log to file. Ideally the files should be able to be opened in Sonus LX tool so that further message debugging can be done easily.
• The syslog server needs to be able to roll the log files once they get to a specific size (so they can be emailed, etc).
• The syslog server should only keep a specific number of these log files so that the server’s hard disk does not get filled with log files.
• Both the display and log files should be able to be filtered to display only information that I want to see. For example, only show lines with a specific phone number in them, or only show me SIP messages. These filters should be independent so that you can view the filtered information on screen whilst more detailed information is getting logged to file for further review and troubleshooting later.

Based on these requirements I figured it would be very cool to write the server in Powershell, as this allows for absolutely no installation and can be run on any Windows machine you are likely to run into. How hard could it be?

<Insert training montage>

SMASH CUT:

EXT. TRAINING MONTAGE - THE STAIRS AT THE FRONT OF THE PHILADELPHIA MUSEUM OF ART- DAY

A man in a sweaty hoody runs to the top of a large set of stairs carrying a tablet based productivity device that he is furiously typing on. A large group of the town’s population is also running after him in a large pack for no apparent reason. Upon reaching the top of the steps he punches the air and launches the tablet into the sky. The tablet hits the concrete and smashes into a million pieces. He falls to the ground and screams towards the sky.

MAN

Nooooooo! I should have backed up to the cloud, the cloud I tells ya.

Okay okay, let’s cut to the chase. I did it, and now you too can syslog with me into the sunset.

Power Syslog Server

Features:

• Zero installation.
• Signed Powershell Script.
• Real time log display (Approximately 1000 lines).
• Copy the displayed text with the Copy Text button. This is useful for more in depth analysis in your favourite notepad software.
• Rolling log files based on file size and number of files to keep.
• Clear display and Pause display functions.
• Filter real-time display logging with regular expression.
• Filter logging to file with regular expression.
• Log files can be opened in Sonus LX tool for further debugging.
• Open firewall for Syslog Server port with the click of a button. If you are not seeing any syslog output in the Power Syslog Server display log then try pressing the Open Firewall button.
• Server listening port can be changed by creating a config file (PowerSyslogServerSettings.cfg) in the same directory as the script. The config file needs to have text in it in the following format "SysLogPort=514". This allows you to maintain the integrity of the code signing by not directly editing the script file.

Download 1.0

How to configure a Sonus Gateway for Syslog Output

Sonus makes some of the most popular Lync Gateways on the market, so I have chosen to use them as an example of how to set up a device to output syslog. Power Syslog Server will work with any other UDP based syslog client as well though, so feel free to use it with other devices too.

Remote Log Servers:

Setup your device to output syslog to the server you are running Power Syslog Server on.  

Global Log Level: If your subsystems are set to “Default” logging level then this setting will be applied to them. This is also the level it will log for all services that are not specified in Subsystems. You will usually set this to a low value like “Error” or “Warning” to avoid log flooding.

Log Destination: The server with the Power Syslog Server running on it.

Port: 514                 

Protocol: UDP

Log Facility: local0

Enabled: Yes

Important Note: When you're finished debugging remember to Disable the syslog output. Otherwise the device will continue to output syslog data over the network, which can be a significant amount of unnecessary overhead for your device, network and server. 

Subsystems:

Then enable the Subsystems as required:

Subsystem: Set the specific Subsystem that you would like to have logged to the syslog output. For troubleshooting call flows and SIP messaging the “SIP Stack Service”, “Common Call Control” (for ISDN translation tables), “Call Routing Service” (for SIP translation tables), and "ISDN Protocol" (for E1 integrations) are useful subsystems to configure here.

Log Level: Set the required Log Level.

Log Destination:The Remote Log Server we created in the first step.

Debugging Log Files in LX Tool

Once you have captured your syslog files using the Power Syslog Server on the server on site you may want to do further call flow debugging using the Sonus LX tool (which can offer you decoded call flows for both SIP and ISDN calls providing your syslog contrains "ISDN Protocol" DEBUG and "SIP Stack Service" DEBUG logging).

To import the file into the LX tool, simply take one of the log files that the Power Syslog Server created and drag it into the LX tool window (or use File->Open). When you do this the LX tool will break the syslog file down into the individual call flows that were captured in the log. Here is an example:

Sonus LX Tool

By double clicking on a call in the "Calls" tab at the bottom of the screen you can get further details on each call flow (including ISDN decoding!):

Sonus LX Tool - Call Flow

Note: The LX Tool is a tool orginally created by NET (which was subsequently aquired by Sonus). To get a copy of the software go to the Sonus Salesforce portal and select "Software Downloads" then select "LX" from the Products list. If you don't have access to the Portal, speak to your Sonus representative to get a copy of the software.

Example Display/Log Filters

Power Syslog Server includes a cool feature that allows you to filter (using regular expressions) what lines of syslog get displayed on the screen and logged to file. The reason for allowing for having a separate Display Filter and Log Filter is to help you when troubleshooting in real time. By this I mean that you can configure a very specific Display Filter to allow you to see only the messages you want to see for a specific issue and a more general Log File Filter so you can capture more detailed logs to review later in order to pinpoint the exact cause of the issue. Below are some examples of how you can use these filters when troubleshooting issues:

Show Only SIP Messaging

When you are running SIP Stack Service logging at a DEBUG level the Sonus gateway will output all of the SIP messaging that is traversing it. This can be very useful when you need to know what error messages are being sent by the Carrier SIP network or Lync when a call fails.

Example Filter (without quote marks): “sip:”

Example Output:

192.168.0.20 <135>[2014-09-16 00:57:02,709]  287 0002

OPTIONS sip:ux1000lab.mylynclab.com SIP/2.0

FROM: <sip:2013ENTFE003.mylynclab.com:5068;transport=Tcp;ms-opaque=152721d992435f69>;epid=B3F80C5FC7;tag=fb568a1fab

TO: <sip:ux1000lab.mylynclab.com>

CSEQ: 9993 OPTIONS

CALL-ID: 87a0bbd93e7f4e33a2c87ff8bbccd3d7

MAX-FORWARDS: 70

VIA: SIP/2.0/TCP 192.168.0.96:51823;branch=z9hG4bK96df5daa

CONTACT: <sip:2013ENTFE003.mylynclab.com:5068;transport=Tcp;maddr=192.168.0.96>

CONTENT-LENGTH: 0

USER-AGENT: RTCC/5.0.0.0 MediationServer

192.168.0.20 <135>[2014-09-16 00:57:02,718]  322 0001

SIP/2.0 200 OK

Allow: INVITE, ACK, CANCEL, BYE, UPDATE, NOTIFY, OPTIONS, REFER, REGISTER

Call-ID: 87a0bbd93e7f4e33a2c87ff8bbccd3d7

Content-Length: 0

CSeq: 9993 OPTIONS

From:  <sip:2013ENTFE003.mylynclab.com:5068;transport=Tcp;ms-opaque=152721d992435f69>;epid=B3F80C5FC7;tag=fb568a1fab

Server: SONUS SBC1000 3.0.2v270 Sonus SBC

Supported: replaces,update,100rel

To:  <sip:ux1000lab.mylynclab.com>;tag=aedb006-3ef64

Via: SIP/2.0/TCP 192.168.0.96:51823;branch=z9hG4bK96df5daa

192.168.0.20 <135>[2014-09-16 00:57:04,827]  393 0003

OPTIONS sip:siptrunk.aapt.com.au:5060 SIP/2.0

Allow: INVITE, ACK, CANCEL, BYE, UPDATE, NOTIFY, OPTIONS, REFER, REGISTER

Call-ID: call-71280200-0000-0010-1101-0@10.237.176.6

Content-Length: 0

CSeq: 132654 OPTIONS

From:  <sip:Anonymous@10.237.176.6:5060>;tag=aedb006-1

Max-Forwards: 70

Supported: replaces,update,100rel

To:  <sip:Anonymous@siptrunk.aapt.com.au:5060>

User-Agent: SONUS SBC1000 3.0.2v270 Sonus SBC

Via: SIP/2.0/UDP 10.237.176.6:5060;branch=z9hG4bK-UX-0aed-b006-40c88

Show Output Relating to Transformation and Route Rules

This can be extremely useful for troubleshooting what transformation rules a call is using and what routing rule it has chosen.

Example Filter (without quote marks): “regex match|transformation|route request”

Note: You need to be logging at DEBUG level for “Common Call Control” (for ISDN translation tables) and the “Call Routing Service” (for SIP translation tables) for this to work.

Example Output:

192.168.0.20 <134>[2014-09-16 00:51:13,126] 1160 0097 com.sonus.sbc.route INFO (callrouter.cpp:2193) - Handling route request.

192.168.0.20 <135>[2014-09-16 00:51:13,127] 1163 0094 com.sonus.sbc.route DEBUG (translation.cpp:1332) - Performing OPTIONAL transformation using entry Testing Calling Party Rule (13.1(4)).

192.168.0.20 <135>[2014-09-16 00:51:13,127] 1164 0093 com.sonus.sbc.route DEBUG (translation.cpp:649) - Failed regex match of "tfCallingSubNumber" field for "^(9999113\d{2})$" (updated "^(9999113\d{2})$") with input of ""

192.168.0.20 <135>[2014-09-16 00:51:13,127] 1165 0092 com.sonus.sbc.route DEBUG (translation.cpp:1332) - Performing OPTIONAL transformation using entry 4 digit to E.164 (13.2(1)).

192.168.0.20 <135>[2014-09-16 00:51:13,127] 1166 0091 com.sonus.sbc.route DEBUG (translation.cpp:653) - Successful regex match of "tfCalledNumber" field for "^(45\d{2})$" (updated "^(45\d{2})$") with input of "4501"

192.168.0.20 <135>[2014-09-16 00:51:13,127] 1168 008f com.sonus.sbc.route DEBUG (translation.cpp:1332) - Performing OPTIONAL transformation using entry Full National to Lync (13.3(2)).

192.168.0.20 <135>[2014-09-16 00:51:13,127] 1169 008e com.sonus.sbc.route DEBUG (translation.cpp:649) - Failed regex match of "tfCalledNumber" field for "^0(3958245\d{2})$" (updated "^0(3958245\d{2})$") with input of "+61395824501"

192.168.0.20 <135>[2014-09-16 00:51:13,127] 1170 008d com.sonus.sbc.route DEBUG (translation.cpp:1332) - Performing OPTIONAL transformation using entry Local to Lync (13.4(3)).

192.168.0.20 <135>[2014-09-16 00:51:13,127] 1171 008c com.sonus.sbc.route DEBUG (translation.cpp:649) - Failed regex match of "tfCalledNumber" field for "^(958245\d{2})$" (updated "^(958245\d{2})$") with input of "+61395824501"

192.168.0.20 <134>[2014-09-16 00:51:13,127] 1172 008b com.sonus.sbc.route INFO (callrouter.cpp:2396) - Successful route request with entry Analog to Lync (5.1(3))

Show Only Syslog Lines Related to a Specific Phone Number

This can be useful if you know a users telephone number and you only want to see messages that relate to them.

Example Filter (without quote marks): “+61399995555”

The Wrap Up

So there you have it, another tool for the kit bag. I hope you like it and find it useful, I know it’s already got me out of a few close calls. If you find any bugs or have any feature requests feel free to drop me a line.

Read more →