SQL SERVER COMMANDS ON BCP, TABLE TO XML, TABLE TO XML -To STRING,TABLE METADATA and other etc

 Bulk capture utility

cmd>BCP
 "select patient_info,patient_id, patient_dft, patient_evn, modifed_date,  from ADT_Patient_stage_Dim where modified_date = 2015-06-17 12:31:48.157"
queryout
 B:\BCP\Export_adt_20150617v1 -SSRDEV1 -UDEVUSER -PDEVLOGIN -c

cmd>BCP  ADT.dbo.ADT_Patient_stage_Dim
 OUT
  B:\BCP\Export_adt_20150617v2  -SSRDEV1 -UDEVUSER -PDEVLOGIN -c


cmd>BCP  ADT.dbo.ADT_Patient_stage_Dim
 IN
  B:\BCP\Export_adt_20150617v2  -SSRDEV1 -UDEVUSER -PDEVLOGIN -c

cmd>BCP ADT.dbo.ADT_Patient_stage_dim
 OUT
  B:\BCP\Export_adt_20150617v3  -T -c

cmd>BCP ADT.dbo.ADT_Patient_stage_dim
 IN
  B:\BCP\Export_adt_20150617v3  -T -c

Explanation of options:
(-T : if the SQL Server instance uses integrated security then use this option for trusted connection, otherwise specify it with -U : loginid, P:password .  )
(-c :The bcp utility uses character data for the bulk copy operation. To bulk copy data between SQL Server and other applications, such as Microsoft Excel.
When you include one of these options, you are not prompted for format information. The formatting is taken care of automatically. Each field, by default, is terminated with a tab character, and each row is terminated with a newline character which can also be changed with -t ,-r options )
(-F argument to specify the first row to be retrieved and the -L argument to specify the last row)
(-o argument to specify an output file,any output that would have been displayed to the console is now saved to the output file.)
(Now when you import the data, any IDENTITY column  in the table values in the data file will be loaded into the table, rather than new values being generated by the database with a -E option)
(-h: command includes the TABLOCK hint, the database engine will hold the lock for the duration of the bulk load operation, which significantly improves performance over the default row-level locks. -h ORDER can also can be used)
(To specify the batch size, include the -b argument, along with the number of rows per batch.)
(To specify an error file, use the -e argument, with full path specified)
(To create a format file, you must specify format for the mode and nul for the data file. In addition, you must also include the -f argument, followed by the full path name of the format file)
(To create an XML format file, you run a command   include the -x argument to create in XML format which is  more flexible and powerful than the original non-XML format file.However, as with the non-XML format file, you can specify a different format "-n:native format" other than "-c:character format")
(The format file, can be referenced in your bcp commands by specifying the -f argument, along with the name of the format file)
Notice: XML format files do not let you have more table columns than fields in the data file. To get around this limitation you have to create an updateable view against the target table and including only those columns that have corresponding fields in the data file.)
 (-N:unicode native format, -W :unicode character format)

BCP with format option


cmd>BCP  ADT.dbo.ADT_Patient_stage_Dim
 fomat nul -c
-f B:\BCP\Export_adt_tabformat.fmt -T

cmd>BCP  ADT.dbo.ADT_Patient_stage_Dim
 fomat nul -c  -x
-f B:\BCP\Export_adt_tabformat.xml -T


Export_adt_tabformat.fmt ,  Export_adt_tabformat.xml are format files exported from table to known the structure of the import dataset to be imported into table

cmd:> BCP  ADT.dbo.ADT_Patient_stage_Dim
   IN
B:\BCP\Hcodition.txt
-f B:\BCP\Export_adt_tabformat.xml -T -c

cmd> BCP  ADT.dbo.ADT_Patient_stage_Dim
   IN
B:\BCP\Hcodition.txt
-f B:\BCP\Export_adt_tabformat.txt -T -c


In query window(from table  to xml)


select @@SERVERNAME. GETDATE();

 select patient_info,patient_id, patient_dft, patient_evn, modifed_date,  from ADT_Patient_stage_Dim where modified_date = '2015-06-17 12:31:48.157'
for xml auto,root('ADTINFO')
GO 

Select(
select patient_info,patient_id, patient_dft, patient_evn, modifed_date,  from ADT_Patient_stage_Dim where modified_date = '2015-06-17 12:31:48.157'
for xml auto,root('ADTINFO')
) AS DATA,
 'B:\BCP\ADTdetails' AS path
GO

For metadata information on tables


DBCC CHECKIDENT(ADT_Patient_stage_Dim,NORESEED) {explain: To know the identity details}

select * from information_schema.tables where TABLE_NAME = 'ADT_Patient_stage_dim'

select * from sys.tables;

select * from sysobjects where xtype ='U'

exec sp_columns  'ADT_Patient_stage_dim'

exec sp_help  'ADT_Patient_stage_dim'


For Change Data Capture


exec sys.sp_cdc_enable_db (TO enable cdc)

Exec sys.sp_cdc_enable_table
    @ Source_Schema = N 'dbo' ,
     @Source_Name = N 'ADT_Patient_stage' ,
    @ Role_Name = N 'cdc_user_tt' ,
    @Supports_NET_changes = 1



TABLE CTAS:


select * into ADT.dbo.ADT_Patient_stage_Dimbkp from  ADT_Patient_stage_Dim

------------------------------------------------------

Which  job_id is using which subsystem(execute in query window)

Use [msdb]
Go

Select
Srv.srvname as ServerName,
Job.name as JobName,
Prgrun.step_id,
Prgrun.step_name as StepName,
Prgrun.command,
Job.enabled,
Prgrun.Subsystem

from dbo.Sysjobs Job

join dbo.Sysjobsteps Prgrun
 on Prgrun.job_id = Job.job_id

join  Master.dbo.sysservers Srv
on Srv.srvid = Job.originating_server_id

where Prgrun.subsystem = 'ssis'

order by Job.name ,step_id

Population Statistics Report using Maps in ssrs

Create a report in SSRS which is a Population.Rdl report with DataSources.Rds and datasets data from a database table .

In SSIS toolbox select the map and drag on it to Designer body(canvas) and in the wizard

STEP 1:select the new map layer from the map gallery or ESRI shape file or SQL server spatial query and in the following steps
STEP 2: crop the map or add a layer
STEP 3: For map visualization select the theme from BASIC MAP,COLOR ANALYTICAL MAP,BUBBLE MAP
STEP 4: Choose the dataset that you created according to the requirement,
STEP 5: Specify the match fields between spatial data and analytical data
STEP 6: Choose the color theme, field to visualize , and color rule which is very important to view the data effects
STEP 7: Right click on polygon properties in the map layer and do customize according to your requirement , The polygon properties and polygon color rule are important to customize

Population stastics report using Multi Group Drill Down Reports-1followed



Using Script Task in SSIS connecting to public government site and downloading the population datasheet into local computer :


#region Help:  Introduction to the script task
/* The Script Task allows you to perform virtually any operation that can be accomplished in
 * a .Net application within the context of an Integration Services control flow.
 *
 * Expand the other regions which have "Help" prefixes for examples of specific ways to use
 * Integration Services features within this script task. */
#endregion


#region Namespaces
using System;
using System.Data;
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;
#endregion
using System.Net;
using System.IO;

namespace ST_7bc92c176223450383db1a3e9c57d929
{
    /// <summary>
    /// ScriptMain is the entry point class of the script.  Do not change the name, attributes,
    /// or parent of this class.
    /// </summary>
    [Microsoft.SqlServer.Dts.Tasks.ScriptTask.SSISScriptTaskEntryPointAttribute]
    public partial class ScriptMain : Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
    {
        #region Help:  Using Integration Services variables and parameters in a script
        /* To use a variable in this script, first ensure that the variable has been added to
         * either the list contained in the ReadOnlyVariables property or the list contained in
         * the ReadWriteVariables property of this script task, according to whether or not your
         * code needs to write to the variable.  To add the variable, save this script, close this instance of
         * Visual Studio, and update the ReadOnlyVariables and
         * ReadWriteVariables properties in the Script Transformation Editor window.
         * To use a parameter in this script, follow the same steps. Parameters are always read-only.
         *
         * Example of reading from a variable:
         *  DateTime startTime = (DateTime) Dts.Variables["System::StartTime"].Value;
         *
         * Example of writing to a variable:
         *  Dts.Variables["User::myStringVariable"].Value = "new value";
         *
         * Example of reading from a package parameter:
         *  int batchId = (int) Dts.Variables["$Package::batchId"].Value;
         * 
         * Example of reading from a project parameter:
         *  int batchId = (int) Dts.Variables["$Project::batchId"].Value;
         *
         * Example of reading from a sensitive project parameter:
         *  int batchId = (int) Dts.Variables["$Project::batchId"].GetSensitiveValue();
         * */

        #endregion

        #region Help:  Firing Integration Services events from a script
        /* This script task can fire events for logging purposes.
         *
         * Example of firing an error event:
         *  Dts.Events.FireError(18, "Process Values", "Bad value", "", 0);
         *
         * Example of firing an information event:
         *  Dts.Events.FireInformation(3, "Process Values", "Processing has started", "", 0, ref fireAgain)
         *
         * Example of firing a warning event:
         *  Dts.Events.FireWarning(14, "Process Values", "No values received for input", "", 0);
         * */
        #endregion

        #region Help:  Using Integration Services connection managers in a script
        /* Some types of connection managers can be used in this script task.  See the topic
         * "Working with Connection Managers Programatically" for details.
         *
         * Example of using an ADO.Net connection manager:
         *  object rawConnection = Dts.Connections["Sales DB"].AcquireConnection(Dts.Transaction);
         *  SqlConnection myADONETConnection = (SqlConnection)rawConnection;
         *  //Use the connection in some code here, then release the connection
         *  Dts.Connections["Sales DB"].ReleaseConnection(rawConnection);
         *
         * Example of using a File connection manager
         *  object rawConnection = Dts.Connections["Prices.zip"].AcquireConnection(Dts.Transaction);
         *  string filePath = (string)rawConnection;
         *  //Use the connection in some code here, then release the connection
         *  Dts.Connections["Prices.zip"].ReleaseConnection(rawConnection);
         * */
        #endregion


        /// <summary>
        /// This method is called when this script task executes in the control flow.
        /// Before returning from this method, set the value of Dts.TaskResult to indicate success or failure.
        /// To open Help, press F1.
        /// </summary>
        public void Main()
        {
            // TODO: Add your code here


            string filePath ="C:\\Users\\simisam\\Desktop\\DataSetsForssis\\apidata\\sampledat1.xml" ;
          /* 

            WebClient client = new WebClient ();
            client.DownloadFileAsync(new Uri("https://api.census.gov/data/2015/pep/population?get=POP,GEONAME&for=region:*&DATE=8&key=08214dffd18610892ee0e9561a5f27f4ef872ad9"), Environment.GetFolderPath(Environment.SpecialFolder.Desktop) + "File.zip");*/


            WebRequest myRequest = WebRequest.Create("http://api.census.gov/data/2015/pep/population?get=POP,GEONAME&for=state:*&key=08214dffd18610892ee0e9561a5f27f4ef872ad9");
            using (WebResponse myResponse = myRequest.GetResponse())
            using (StreamReader reader = new StreamReader(myResponse.GetResponseStream()))
            {
                // use whatever method you want to save the data to the file...
                File.AppendAllText(filePath, myResponse.Headers.ToString());
                File.AppendAllText(filePath, reader.ReadToEnd());
            }



            Dts.TaskResult = (int)ScriptResults.Success;
        }

        #region ScriptResults declaration
        /// <summary>
        /// This enum provides a convenient shorthand within the scope of this class for setting the
        /// result of the script.
        ///
        /// This code was generated automatically.
        /// </summary>
        enum ScriptResults
        {
            Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
            Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
        };
        #endregion

    }
}
As the data is loaded into local computer in xml file format.
 In CONTROL FLOW --> DATA FLOW TASK --> Use XML SOURCE-->EXCEL DESTINATION
Comma separated excel destination 
finally load the 
 EXCEL SOURCE -->DATA CONVERSIONS--> OLE DB DESTINATION

Population stastics report using Multi Group Drill Down Reports-2followed


Population stastics report using Multi Group Drill Down Reports-3followed


Population stastics report using Multi Group Drill Down Reports-4followed


Population stastics report using Multi Group Drill Down Reports-5followed

Using Script Task in SSIS connecting to public government site and downloading the population datasheet into local computer :

#region Help:  Introduction to the script task
/* The Script Task allows you to perform virtually any operation that can be accomplished in
 * a .Net application within the context of an Integration Services control flow.
 *
 * Expand the other regions which have "Help" prefixes for examples of specific ways to use
 * Integration Services features within this script task. */
#endregion


#region Namespaces
using System;
using System.Data;
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;
#endregion
using System.Net;
using System.IO;

namespace ST_7bc92c176223450383db1a3e9c57d929
{
    /// <summary>
    /// ScriptMain is the entry point class of the script.  Do not change the name, attributes,
    /// or parent of this class.
    /// </summary>
    [Microsoft.SqlServer.Dts.Tasks.ScriptTask.SSISScriptTaskEntryPointAttribute]
    public partial class ScriptMain : Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
    {
        #region Help:  Using Integration Services variables and parameters in a script
        /* To use a variable in this script, first ensure that the variable has been added to
         * either the list contained in the ReadOnlyVariables property or the list contained in
         * the ReadWriteVariables property of this script task, according to whether or not your
         * code needs to write to the variable.  To add the variable, save this script, close this instance of
         * Visual Studio, and update the ReadOnlyVariables and
         * ReadWriteVariables properties in the Script Transformation Editor window.
         * To use a parameter in this script, follow the same steps. Parameters are always read-only.
         *
         * Example of reading from a variable:
         *  DateTime startTime = (DateTime) Dts.Variables["System::StartTime"].Value;
         *
         * Example of writing to a variable:
         *  Dts.Variables["User::myStringVariable"].Value = "new value";
         *
         * Example of reading from a package parameter:
         *  int batchId = (int) Dts.Variables["$Package::batchId"].Value;
         * 
         * Example of reading from a project parameter:
         *  int batchId = (int) Dts.Variables["$Project::batchId"].Value;
         *
         * Example of reading from a sensitive project parameter:
         *  int batchId = (int) Dts.Variables["$Project::batchId"].GetSensitiveValue();
         * */

        #endregion

        #region Help:  Firing Integration Services events from a script
        /* This script task can fire events for logging purposes.
         *
         * Example of firing an error event:
         *  Dts.Events.FireError(18, "Process Values", "Bad value", "", 0);
         *
         * Example of firing an information event:
         *  Dts.Events.FireInformation(3, "Process Values", "Processing has started", "", 0, ref fireAgain)
         *
         * Example of firing a warning event:
         *  Dts.Events.FireWarning(14, "Process Values", "No values received for input", "", 0);
         * */
        #endregion

        #region Help:  Using Integration Services connection managers in a script
        /* Some types of connection managers can be used in this script task.  See the topic
         * "Working with Connection Managers Programatically" for details.
         *
         * Example of using an ADO.Net connection manager:
         *  object rawConnection = Dts.Connections["Sales DB"].AcquireConnection(Dts.Transaction);
         *  SqlConnection myADONETConnection = (SqlConnection)rawConnection;
         *  //Use the connection in some code here, then release the connection
         *  Dts.Connections["Sales DB"].ReleaseConnection(rawConnection);
         *
         * Example of using a File connection manager
         *  object rawConnection = Dts.Connections["Prices.zip"].AcquireConnection(Dts.Transaction);
         *  string filePath = (string)rawConnection;
         *  //Use the connection in some code here, then release the connection
         *  Dts.Connections["Prices.zip"].ReleaseConnection(rawConnection);
         * */
        #endregion


        /// <summary>
        /// This method is called when this script task executes in the control flow.
        /// Before returning from this method, set the value of Dts.TaskResult to indicate success or failure.
        /// To open Help, press F1.
        /// </summary>
        public void Main()
        {
            // TODO: Add your code here


            string filePath ="C:\\Users\\simisam\\Desktop\\DataSetsForssis\\apidata\\sampledat1.xml" ;
          /* 

            WebClient client = new WebClient ();
            client.DownloadFileAsync(new Uri("https://api.census.gov/data/2015/pep/population?get=POP,GEONAME&for=region:*&DATE=8&key=08214dffd18610892ee0e9561a5f27f4ef872ad9"), Environment.GetFolderPath(Environment.SpecialFolder.Desktop) + "File.zip");*/


            WebRequest myRequest = WebRequest.Create("http://api.census.gov/data/2015/pep/population?get=POP,GEONAME&for=state:*&key=08214dffd18610892ee0e9561a5f27f4ef872ad9");
            using (WebResponse myResponse = myRequest.GetResponse())
            using (StreamReader reader = new StreamReader(myResponse.GetResponseStream()))
            {
                // use whatever method you want to save the data to the file...
                File.AppendAllText(filePath, myResponse.Headers.ToString());
                File.AppendAllText(filePath, reader.ReadToEnd());
            }



            Dts.TaskResult = (int)ScriptResults.Success;
        }

        #region ScriptResults declaration
        /// <summary>
        /// This enum provides a convenient shorthand within the scope of this class for setting the
        /// result of the script.
        ///
        /// This code was generated automatically.
        /// </summary>
        enum ScriptResults
        {
            Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
            Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
        };
        #endregion

    }
}

As the data is loaded into local computer in xml file format.
 In CONTROL FLOW --> DATA FLOW TASK --> Use XML SOURCE-->EXCEL DESTINATION
Comma separated excel destination
 
finally load the

 EXCEL SOURCE -->DATA CONVERSIONS--> OLE DB DESTINATION